system

Abstract

We provide the system. [Solution] An acquisition means having the function of acquiring image information and identifying waste based on said image information, A classification means having the function of classifying waste based on the identification result of the waste, A reading means having the function of reading identification tags attached to waste and obtaining lifecycle information of the waste, An analysis means having the function of analyzing data based on the lifecycle information and identification results and generating improvement suggestions for a specific manufacturer, A display means that has the function of displaying waste classification information on the user's device and displaying the status of eco-points in real time, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern waste management, there are problems such as the difficulty of appropriate classification according to the type of waste, effectively tracking the life cycle information of waste, and efficiently making continuous improvement proposals to manufacturers. In particular, as the types of waste diversify and it becomes difficult for consumers to accurately classify, there are concerns about a decrease in the recycling rate and an increase in environmental load. Also, there is a problem in that there is a lack of a mechanism to utilize waste life cycle data to reflect it in the product improvement of manufacturers.

Means for Solving the Problems

[0005] This invention improves the accuracy of waste classification by providing a system equipped with acquisition and reading means that identify waste based on image information and acquire lifecycle information using identification tags. Furthermore, by providing analysis means that analyze the data based on this acquisition and identification and automatically generate specific improvement suggestions for particular manufacturers, it enables consistent improvements from waste management to product design optimization. In addition, it incorporates an eco-point function that awards points to consumers who perform correct classification actions, with the aim of raising consumer awareness of sorting and promoting recycling activities.

[0006] "Acquisition means having the function of acquiring image information and identifying waste based on said image information" refers to a device or method that collects image data of waste using a camera or sensor and analyzes and identifies the type of waste based on said data.

[0007] "Classification means having the function of classifying waste based on the identification results of the waste" refers to a device or method that automatically distributes waste into predetermined categories based on the acquired identification results.

[0008] "A reading means having the function of reading identification tags attached to waste and acquiring lifecycle information of waste" refers to a device or method that uses electronic tags such as RFID to read product history and recycling information and acquire lifecycle data of waste.

[0009] "Analysis means having the function of analyzing data based on lifecycle information and identification results and generating improvement suggestions for a specific manufacturer" refers to an apparatus or method that analyzes collected data and automatically creates suggestions for improving products or manufacturing processes.

[0010] The "function that awards points to users who correctly classify waste based on acquired data" is a system that provides incentives such as eco-points to consumers who accurately separate their waste.

[0011] The "function for transmitting generated improvement suggestions to the relevant manufacturers or personnel using designated communication methods" refers to a system that notifies manufacturers or relevant personnel of the generated suggestions via email, a dedicated portal, or the like. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0013] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0018] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This invention is an automated classification system aimed at improving the efficiency of waste management, and mainly consists of acquisition means, classification means, reading means, and analysis means. This system assists users, such as households and businesses, in properly classifying waste.

[0034] First, the device incorporates sensors into the smart trash can and captures image data with a camera when waste is placed inside. This means that waste information is digitized without the user's awareness each time they throw away trash. The images captured by the camera are immediately sent to a server for analysis.

[0035] Next, the server processes the received image data and uses its built-in machine learning algorithms to identify the type of waste. This process allows for classification into categories such as "plastic," "paper," and "metal." The identified data contributes to further analysis of waste trends and recycling potential.

[0036] The terminal also reads identification tags attached to waste using an RFID reader to obtain product lifecycle information. This information includes details such as the manufacturing date and manufacturer, which helps in product tracking and improving the recycling process.

[0037] The server then integrates and analyzes the collected identification information and lifecycle data. This analysis makes it possible to determine the extent to which specific products are recycled and to identify products that need improvement. Based on the analysis results, the system automatically generates a report suggesting product improvements for specific manufacturers.

[0038] Furthermore, the system awards eco-points to users who correctly classify their waste. These eco-points are managed through a separate application or web portal, and users can use them for various purposes. This system motivates users to actively classify their waste correctly and contribute to recycling efforts.

[0039] It is expected that the implementation of this invention will lead to efficient waste management and improved recycling rates, thereby contributing to the formation of a sustainable society.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The device uses sensors to detect when waste is placed in the smart trash can. When the sensor is triggered, the camera automatically activates and takes a picture of the waste. At this stage, the sensor does not detect the type or amount of waste; it simply detects the act of placing waste in the trash can.

[0043] Step 2:

[0044] The terminal transmits the acquired image data to the server in real time. The images are converted to a standard format and programmed to prevent errors. This ensures that the analysis process can begin without delay.

[0045] Step 3:

[0046] The server inputs the received image data into an AI image recognition module to identify the type of waste. The AI ​​uses a multi-layer neural network to classify objects into categories such as "paper," "plastic," and "metal" based on their shape and structure. This model has been pre-trained on a vast dataset, resulting in high accuracy.

[0047] Step 4:

[0048] The terminal reads the RFID tags in the waste using an RFID reader. The information obtained from these tags includes product IDs and detailed manufacturing information, which is then transmitted to a server, allowing for accurate identification of the waste's characteristics.

[0049] Step 5:

[0050] The server integrates the collected image recognition data and RFID data, and performs analysis by querying it against information stored in the database. This analysis helps determine how frequently specific types of waste are being introduced and how much room there is for improvement, thereby generating suggestions for product improvements.

[0051] Step 6:

[0052] Users earn eco-points by properly classifying their waste. These points are managed through a smartphone application, and accumulated points can be used for discount coupons or product purchases, contributing to increased environmental awareness among users.

[0053] Step 7:

[0054] The server automatically generates improvement suggestions based on the analysis results and notifies designated manufacturers and stakeholders. These suggestions are sent via email or a web portal and used as feedback for product improvement.

[0055] (Example 1)

[0056] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0057] In modern society, vast amounts of waste are generated, and its proper management has a significant impact on the global environment. However, effective waste classification and improved recycling rates require active participation from users, as well as product improvement suggestions from manufacturers. Given this situation, there is a lack of systems to promote efficient waste management and recycling activities.

[0058] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0059] In this invention, the server includes a device having the function of acquiring image information and identifying organic matter based on the image information, a device having the function of classifying substances based on the identification results of the organic matter, and a device having the function of reading identification tags attached to substances and acquiring usage history information of the substances. This enables users to accurately classify waste without being aware of it, generates appropriate improvement suggestions for producers, and promotes the efficiency of waste management and recycling activities.

[0060] "Image information" refers to digital data that visually captures the appearance of a substance and is used for identification and classification.

[0061] "Organic matter" refers to substances that include materials derived from living organisms and their processed products, and are classified as recyclable resources.

[0062] An "identification tag" is a small identification device attached to an object that has the function of electronically storing information such as usage history.

[0063] "Usage history information" refers to a series of pieces of information about a substance from its manufacture to its use and disposal, and is data used to understand the product's lifecycle.

[0064] A "device" refers to a combination of hardware and software designed to perform a specific function.

[0065] A "producer" refers to a corporation or individual responsible for manufacturing or supplying a product, and is the party to whom suggestions regarding product improvements should be received.

[0066] This invention is a system for achieving efficient waste management through the cooperation of a terminal, a server, and a user. Specifically, the terminal is installed in a smart trash can and is equipped with a sensor that activates when waste is placed inside and a camera for capturing images. This camera acquires image information of the waste and transmits that data to the server in real time. Through this operation, the waste is digitized without the user having to be particularly aware of it.

[0067] The server functions as a device that executes machine learning algorithms using the received image information. For example, it applies known deep learning models such as ResNet and VGG to identify the type of waste and classify it into categories such as "plastic," "paper," and "metal" by comparing it with a database. Furthermore, the terminal uses an RFID reader to read the identification tags attached to the materials and obtain information on the material's usage history. This provides information for a detailed analysis of the product's lifecycle.

[0068] The collected identification and usage history information is integrated on a server, and improvement suggestions are generated for specific producers. These suggestions include specific improvement plans aimed at increasing the efficiency of waste recycling and are communicated to producers or relevant parties through a predetermined communication method. This facilitates improvements during the product development stage.

[0069] Users are rewarded for correctly classifying their waste, and these rewards can be managed through a dedicated application or web portal. Users can use these rewards to exchange for benefits within their local community.

[0070] As a concrete example, when a user places a plastic bottle into a smart trash can, the device takes a picture of it with its camera, and the image sent to the server is analyzed and classified as recyclable "plastic." The RFID tag attached to the plastic bottle is read, and details such as "Manufacturer: General Incorporated Association, Manufacturing Date: Specified Date" are obtained, and the product's recycling data is updated based on this information. An example of a prompt to be input into the generating AI model would be, "Explain how paper waste is identified in the automated waste classification system. Include the name of the relevant machine learning algorithm and processing procedure."

[0071] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0072] Step 1:

[0073] The device uses sensors to detect when waste is placed in the smart trash can. This detection triggers a camera to activate, taking an image of the waste and generating image data. The input is the physical placement of waste, and the output is digital image data. The generated image data is immediately sent to the server.

[0074] Step 2:

[0075] The server receives image data from the terminal. Next, it inputs the image data into a machine learning algorithm to identify the type of waste. This process utilizes generative AI models such as ResNet and VGG to analyze image features and classify the waste into categories such as "plastic," "paper," and "metal." The input is image data, and the output is the category information of the classified waste.

[0076] Step 3:

[0077] The terminal reads the identification tag attached to the waste using an RFID reader. This retrieves product lifecycle data, such as manufacturer and manufacturing date. The input is the RFID tag attached to the waste, and the output is the usage history information contained within the tag. This information is sent to a server for product tracking and analysis.

[0078] Step 4:

[0079] The server integrates the category information of classified waste with usage history information obtained from RFID readers. Based on this information, it performs analysis to identify the recycling rate of products and areas for improvement at a specific producer. Based on the analysis results, it automatically generates improvement suggestions. The input is integrated information, and the output is improvement suggestions for producers.

[0080] Step 5:

[0081] When a user correctly sorts waste, the server calculates and awards eco-points as a corresponding reward. These points are managed through a dedicated application or web portal, and users can use them to exchange for local benefits. Inputs are the sorting results and user ID, and output is the awarded eco-point information.

[0082] Through these steps, the system enables efficient waste management and supports active user participation and product improvement.

[0083] (Application Example 1)

[0084] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0085] Proper waste classification is crucial for urban environments and business operations, but the cumbersome process of individual users consciously classifying waste leads to misdisposal and low recycling rates. Furthermore, insufficient user motivation to promote recycling activities necessitates more efficient waste management. Another challenge is the lack of timely feedback to manufacturers regarding product improvements.

[0086] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0087] In this invention, the server includes an acquisition means for acquiring image information and identifying waste, a classification means for classifying waste based on the identification results, a reading means for acquiring and analyzing lifecycle information, and a presentation means for displaying waste classification information and eco-points on the user's device. This enables real-time waste classification, effective utilization of an eco-point system, and product improvement through rapid feedback.

[0088] "Acquisition means" refers to a device that has the function of acquiring image information related to waste and performing identification based on that data.

[0089] A "classification device" is a device that analyzes acquired image information of waste and classifies it into the appropriate category.

[0090] A "reading device" is a device that has the function of reading identification tags attached to waste and obtaining its lifecycle information.

[0091] The "analysis means" is a device that performs an analysis by comprehensively analyzing the acquired lifecycle information and identification information, and has the function of generating product improvement suggestions for manufacturers.

[0092] A "presentation means" is a device that has the function of visually displaying waste classification information and eco-point status in real time on the user's device.

[0093] "Eco-points" are points awarded to users when they properly classify and dispose of waste, and the system is designed to incentivize users' environmental protection activities.

[0094] The system implementing this invention consists of three components: a server, a terminal, and a user. For the entire system to function smoothly, each of these components must fulfill its respective role. First, the terminal uses sensors and cameras installed in the smart trash can to acquire image data when waste is placed inside. Specifically, the camera takes high-resolution images of the waste, and the RFID reader reads the identification tag attached to the waste. This information is immediately transmitted to the server.

[0095] The server uses image recognition libraries such as OpenCV and TENSORFLOW® to process the received image data. This classifies the waste into categories such as "plastic," "paper," and "metal." Next, the server stores the read lifecycle information in a database and performs analysis. The analysis results are used to evaluate the traceability and recyclability of the product. Furthermore, based on the analysis data, the server awards eco-points to users who correctly classify their waste.

[0096] Users can check eco-points and waste classification information in real time using their smartphones or smart glasses. The system visually displays the classification results and suggestions for using eco-points on these devices. For example, if a user correctly recycles plastic, a message such as "Your plastic has been correctly recycled. Eco-points have been awarded." will be displayed.

[0097] As a concrete example, when a user disposes of waste at home, using smart glasses would provide real-time information such as, "This is metal. Recycling it will increase your eco-points." Such a system would create an incentive for users to participate more actively in recycling activities.

[0098] Furthermore, an example of a prompt message from the AI ​​model in this system is a text-based message such as, "Based on the photo and RFID information of this waste, explain how to classify it and calculate and present the appropriate eco-points." This incentivizes the server to perform appropriate classification and rewarding.

[0099] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0100] Step 1:

[0101] The terminal uses a camera mounted on the smart trash can to acquire image data when waste is placed inside. This acquired image becomes the input. The input image data is immediately transmitted from the terminal to the server for identification. As preparation for transmission, a specific operation is performed to convert the image data format.

[0102] Step 2:

[0103] The server identifies the type of waste from the received image data. This process uses image recognition libraries such as OpenCV or TensorFlow, with the image data as input. The server performs pattern analysis on the data and obtains output classified into categories such as "plastic," "metal," and "paper."

[0104] Step 3:

[0105] The terminal uses an RFID reader to read the identification tag attached to the waste. The reading result is sent to the server as input. Specifically, this involves decoding the tag information and saving it to a database.

[0106] Step 4:

[0107] The server performs analysis based on the lifecycle information and identification results. It receives lifecycle information and identification results as input, performs data analysis, and outputs an optimized recycling method. These results are used for future improvements and generate feedback reports for manufacturers.

[0108] Step 5:

[0109] The server calculates and awards eco-points for waste correctly classified by the user. The server uses the classification results as input, performs calculations to award eco-points, and outputs them as the user's score. Specifically, point records and account integration are performed.

[0110] Step 6:

[0111] Users can check waste classification information and eco-point status through an application on their device. Based on output data transmitted from the server, the device visualizes and presents the information to the user. Specifically, it performs real-time information updates and notifications.

[0112] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0113] This invention is a system that combines acquisition means, classification means, reading means, analysis means, and emotion recognition engine to improve the efficiency of waste management. This system assists the user in properly classifying waste while simultaneously analyzing the user's emotional state and providing feedback and adjusting the process.

[0114] First, the device incorporates sensors into the smart trash can to detect when waste is placed inside. At this time, a camera is used to acquire image data. This system automatically collects data every time the user disposes of waste. This image data is transmitted to a server in real time, ready for analysis.

[0115] The server processes the received image data using an AI image recognition module to identify the type of waste. The identified data is used not only for waste classification but also for overall analysis. The terminal also uses an RFID reader to obtain lifecycle data from identification tags attached to the waste and transmits this data to the server.

[0116] Furthermore, the server uses an emotion recognition engine to analyze the user's facial expressions and voice tone captured by the camera to identify the user's emotional state. Based on this emotional data, the system optimizes the interface and feedback provided to the user. For example, if it determines that the user is experiencing stress during waste sorting, it provides guidance and assistance to make the process more intuitive.

[0117] Furthermore, users who properly sort their waste will be awarded eco-points. These points serve as a means to raise users' environmental awareness and promote recycling activities. Eco-points are managed through a dedicated application and can be used for various purposes.

[0118] Ultimately, the server generates suggestions for product improvement and environmental impact reduction based on all identification and analysis data, and notifies the relevant manufacturers and responsible parties. This notification process is carried out quickly and efficiently via email and a web portal.

[0119] Thus, by implementing the present invention, it is expected that user convenience in waste management will be enhanced and that the realization of a sustainable society will be contributed to.

[0120] The following describes the processing flow.

[0121] Step 1:

[0122] The terminal uses sensors to detect when waste is dumped. When waste is discarded, the camera automatically activates and takes an image. This image records the appearance of the discarded waste. The captured image data is sent to a server for analysis.

[0123] Step 2:

[0124] The server passes the received image data to an AI image recognition module, which analyzes the type of waste. For example, it is classified into categories such as "paper," "plastic," and "metal." This classification information is used to support the proper disposal of the waste.

[0125] Step 3:

[0126] The terminal uses an RFID reader to read identification tags attached to waste. Manufacturer and product information is obtained from these tags and sent to a server as lifecycle data. This makes it possible to track the origin and characteristics of the waste.

[0127] Step 4:

[0128] The server analyzes the identification data and lifecycle information of the collected waste. Specifically, it performs trend analysis and automatically generates improvement suggestions for manufacturers. This provides concrete measures for manufacturers to reduce their environmental impact.

[0129] Step 5:

[0130] The server uses an emotion recognition engine to analyze the user's facial expressions and voice patterns to assess their emotional state. For example, if the user is stressed, the server adjusts the interface to simplify operations and improve the user experience.

[0131] Step 6:

[0132] Users are awarded eco-points for properly classifying their waste. These points are managed through a dedicated application, allowing for easy checking and use. Eco-points can be used for discounts on reusable products and other benefits, increasing users' contribution to the environment.

[0133] Step 7:

[0134] Based on the analysis and sentiment recognition results, the server notifies the relevant manufacturer or person in charge of product improvement suggestions. Notifications are made via email or a dedicated portal, creating a rapid feedback loop. This facilitates continuous product improvement.

[0135] (Example 2)

[0136] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0137] In waste management, it is necessary to efficiently identify and classify waste appropriately, provide feedback that takes user sentiment into consideration, and promote recycling activities. Furthermore, it is crucial to use the acquired data to drive product improvements in the manufacturing sector. Current systems suffer from insufficient waste identification and classification, as well as user support, leading to delays in suggestions and notifications to the manufacturing sector.

[0138] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0139] In this invention, the server includes an acquisition means having the function of acquiring image data and identifying waste based on the image data; a reading means having the function of reading information tags associated with the waste and acquiring waste usage history information; and an analysis means having the function of identifying the user's emotional state based on camera images and audio data and optimizing the interface and feedback content based on the emotional state. This enables appropriate identification and classification of waste, provides appropriate feedback according to the user's emotional state, and facilitates rapid improvement suggestions in the manufacturing field.

[0140] "Image data" refers to visual information acquired to represent the appearance of waste.

[0141] "Acquisition means" refers to a device or function for collecting image data.

[0142] "Classification means" refers to a device or function for dividing identified waste into specific categories.

[0143] "Reading means" refers to a device or function for detecting information tags associated with waste and collecting that information.

[0144] An "information tag" is an identifier attached to waste materials to record information about their usage history.

[0145] "Usage history information" refers to information about the waste's manufacturer, usage route, and recovery status.

[0146] "Analysis means" refers to a device or function for analyzing the condition of waste and users based on acquired data.

[0147] "Emotional state" refers to information that represents the user's emotions and is identified through facial recognition and voice analysis.

[0148] An "interface" is a mechanism or method for a user to interact with a system.

[0149] "Feedback" refers to information and instructions provided to a user, used to support their actions.

[0150] An "improvement suggestion" is advice or recommendation based on data analysis to encourage improvements to products and processes in the manufacturing sector.

[0151] "Network infrastructure" refers to the communication infrastructure used to transmit data.

[0152] This invention is a system for improving the efficiency of waste management, and includes acquisition means, classification means, reading means, analysis means, and feedback optimization function. When a user disposes of waste, this system acquires image data and audio data, and identifies and analyzes the type of waste based on that data. Furthermore, it analyzes the user's emotional state and provides feedback tailored to the user.

[0153] Specifically, the device incorporates sensors into the smart trash can to detect when waste is placed inside. These sensors include infrared and pressure sensors. Once waste is detected, a camera activates to capture high-resolution images of the waste. This image data is transmitted to a server in real time. The server uses a generative AI model to analyze the image data and identify the type of waste. For example, it may be classified as plastic, metal, or paper. This classification result is stored in a database and used to improve waste management.

[0154] Simultaneously, the terminal reads the information tags attached to the waste and obtains lifecycle data. This data, which includes the product's manufacturer and usage history, is transferred to a server and used for further analysis.

[0155] Furthermore, the server incorporates an emotion recognition engine that uses the camera and voice input capabilities to analyze the user's facial expressions and tone of voice. The emotion data obtained from this analysis is used to optimize the user interface. If the user shows difficulty with waste classification, the system provides assistance and offers instructions to improve the classification process.

[0156] Furthermore, an eco-point system has been introduced, where users are awarded points for correctly classifying their waste. These points not only raise users' environmental awareness but can also be exchanged for rewards. Points are managed using a dedicated application and can be exchanged for things like discount coupons at supermarkets.

[0157] Ultimately, the server generates improvement suggestions based on all identification and sentiment data and notifies managers in the relevant manufacturing sectors. This notification is made via email or a web portal to enable rapid feedback. For example, a prompt for the generating AI model might say, "Suggest ways to optimize the recycling of plastic bottles using a waste management system."

[0158] Thus, this invention improves the efficiency of waste management while simultaneously personalizing the user experience and contributing to the realization of a sustainable society.

[0159] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0160] Step 1:

[0161] The device detects waste being placed in the smart trash can using a sensor built into the trash can. This sensor uses infrared technology and emits a signal when waste passes through the opening. The input is a physical change recognized by the sensor. By detecting this change, an output is obtained that recognizes that waste has been placed in the trash can.

[0162] Step 2:

[0163] The device activates its camera and acquires image data of the waste. Multiple images are taken from different angles and stored as high-resolution visual information. The input is the physical shape and color of the waste, and the output is image data. This data is transmitted to the server in real time.

[0164] Step 3:

[0165] The server analyzes the received image data using a generation AI model. The input is the transmitted image data, and the AI ​​identifies the type of waste based on a deep learning algorithm. This analysis generates waste category information as output. For example, it might be classified as plastic, metal, paper, etc.

[0166] Step 4:

[0167] The terminal reads information tags attached to waste and acquires lifecycle data using radio frequency identification (RFID) technology. The input is the RFID tag, and the output is sent to the server as usage history information. This information provides detailed data regarding the origin and usage route of the waste.

[0168] Step 5:

[0169] The server uses an emotion recognition engine to analyze the user's facial expressions and voice tone captured by the camera. Input includes the user's facial image and voice data. The output identifies the user's emotional state, and the system uses this data to provide corresponding feedback.

[0170] Step 6:

[0171] Based on the user's emotional state, the system optimizes feedback and the interface. Specifically, if it determines that the user is experiencing stress, it displays guidelines to simplify operation or provides voice assistance. The input is the user's emotional data, and the output is optimized user interface information.

[0172] Step 7:

[0173] The system awards eco-points to users who correctly classify their waste. Input includes information about the accuracy of the classification, and the points are recorded in the user's account as output. These points can be managed and viewed through a dedicated application.

[0174] Step 8:

[0175] The server analyzes all collected data and generates improvement suggestions for the manufacturing sector. Inputs include identification results and usage history information, while output lists specific actions as improvement suggestions. These suggestions are then communicated to relevant parties via email and a web portal.

[0176] (Application Example 2)

[0177] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0178] In today's mass-consumption society, proper waste classification and recycling are crucial issues from an environmental protection perspective. However, users often struggle to accurately classify waste, and the process can be stressful. Therefore, there is a need for support systems to improve the efficiency of waste management.

[0179] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0180] In this invention, the server includes an acquisition means having the function of acquiring image information and identifying waste based on said image information; a classification means having the function of classifying the waste based on the identification result of said waste; and an analysis means having the function of analyzing the user's emotional state and optimizing the user interface based on that data. This makes it easier for the user to intuitively classify waste and to provide appropriate feedback to further increase motivation.

[0181] "Image information" refers to visual data acquired through cameras and other imaging devices.

[0182] "Waste" refers to used products and unwanted materials that require proper disposal or recycling.

[0183] "Acquisition means" refers to a function for collecting image information of waste and using it within the system.

[0184] A "classification method" is a function that identifies the type of waste based on the image information acquired and sorts it into the appropriate category.

[0185] The "reading means" refers to a function that scans identification tags attached to waste materials to obtain their lifecycle information.

[0186] "Analysis means" refers to a function that processes data based on lifecycle information and identification results, and generates suggestions for improving the user interface.

[0187] "Emotional state" refers to the psychological state analyzed from the user's facial expressions, voice, etc., and is an element that the system uses to optimize interaction.

[0188] An "interface" is a user interaction function that allows users to interact with a system and exchange information.

[0189] "Eco-points" are rewards given to users who properly classify their waste, and serve as an incentive to promote environmental awareness.

[0190] In the system implementing this invention, a camera installed on a terminal first collects image information of the waste. The acquired image information is sent to a server, where an AI image recognition model is used to identify the type of waste. Specific software used includes PyTorch and TensorFlow. Based on this identification data, the server appropriately classifies the waste.

[0191] The terminal also features an RFID reader to read identification tags attached to waste items, acquiring lifecycle information. The acquired lifecycle information and identification data are analyzed on a server. During this process, since the user's emotional state affects the user interface, an emotion recognition engine is used to analyze the user's facial expressions and voice to identify their state. Based on the analysis results, support information and feedback are provided to help the user process waste more easily.

[0192] Furthermore, users who correctly classify their waste are awarded eco-points, which are managed through a dedicated application. This point system serves as an incentive to raise users' environmental awareness. The server also has the ability to generate suggestions for product improvement and environmental impact reduction based on all identification and analysis data, and notify relevant organizations. These notifications are sent via email and a web portal.

[0193] A concrete example is a system installed at a recycling center in a certain city. When a user disposes of a plastic bottle and records the process through smart glasses, the bottle is immediately identified and the interface is adjusted accordingly. If the user shows signs of confusion based on sentiment analysis, the system provides simple guidance. Furthermore, if the disposal is done properly, eco-points are added to the user's smartphone app, encouraging community use.

[0194] Examples of prompts for a generative AI model:

[0195] "Please explain, with examples, how a smart recycling assistant app can improve residents' waste disposal habits."

[0196] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0197] Step 1:

[0198] The device uses a sensor to detect when waste is placed in a trash can and a camera to capture an image of the waste. This image is immediately sent to a server and processed as image information. The input is image data from the camera, and the output is digital data containing that information.

[0199] Step 2:

[0200] The server analyzes the received image information using an AI image recognition module to identify the type of waste. This process inputs the acquired image data into an AI model and outputs an identification label as a classification result. Specifically, an identification algorithm using TensorFlow is in operation.

[0201] Step 3:

[0202] Upon receiving the identification result, the terminal uses an RFID reader to read the identification tag attached to the waste. It then obtains the waste's lifecycle information from this tag and transmits it to the server. The input is the signal from the identification tag, and the output is the lifecycle information derived from that data.

[0203] Step 4:

[0204] The server activates an emotion recognition engine based on lifecycle information and identification results, and analyzes the user's emotional state using camera and audio data. The input is the user's facial expressions and audio data, and the output is an evaluation of their emotional state. OpenCV and the emotion_recognition library are used in this process.

[0205] Step 5:

[0206] Based on the analyzed data, the server optimizes the user interface and provides support information in real time. If the user is experiencing stress, it presents more user-friendly guides and assistance. The input is emotional evaluation, and the output is an optimized interface and feedback.

[0207] Step 6:

[0208] If a user correctly classifies their waste, the server calculates eco-points and adds them to the user's account. This process calculates the number of points based on the classification results and sends that information to the user's application. The input is the accuracy of the classification, and the output is a notification of point addition.

[0209] Step 7:

[0210] The server integrates all the data and generates suggestions for product improvements and reducing environmental impact. These suggestions are communicated to relevant organizations via email and a web portal. The input is the entire analyzed data, and the output is the notification message for the suggestions.

[0211] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0212] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0213] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0214] [Second Embodiment]

[0215] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0216] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0217] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0218] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0219] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0220] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0221] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0222] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0223] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0224] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0225] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0226] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0227] This invention is an automated classification system aimed at improving the efficiency of waste management, and mainly consists of acquisition means, classification means, reading means, and analysis means. This system assists users, such as households and businesses, in properly classifying waste.

[0228] First, the device incorporates sensors into the smart trash can and captures image data with a camera when waste is placed inside. This means that waste information is digitized without the user's awareness each time they throw away trash. The images captured by the camera are immediately sent to a server for analysis.

[0229] Next, the server processes the received image data and uses its built-in machine learning algorithms to identify the type of waste. This process allows for classification into categories such as "plastic," "paper," and "metal." The identified data contributes to further analysis of waste trends and recycling potential.

[0230] The terminal also reads identification tags attached to waste using an RFID reader to obtain product lifecycle information. This information includes details such as the manufacturing date and manufacturer, which helps in product tracking and improving the recycling process.

[0231] The server then integrates and analyzes the collected identification information and lifecycle data. This analysis makes it possible to determine the extent to which specific products are recycled and to identify products that need improvement. Based on the analysis results, the system automatically generates a report suggesting product improvements for specific manufacturers.

[0232] Furthermore, the system awards eco-points to users who correctly classify their waste. These eco-points are managed through a separate application or web portal, and users can use them for various purposes. This system motivates users to actively classify their waste correctly and contribute to recycling efforts.

[0233] It is expected that the implementation of this invention will lead to efficient waste management and improved recycling rates, thereby contributing to the formation of a sustainable society.

[0234] The following describes the processing flow.

[0235] Step 1:

[0236] The device uses sensors to detect when waste is placed in the smart trash can. When the sensor is triggered, the camera automatically activates and takes a picture of the waste. At this stage, the sensor does not detect the type or amount of waste; it simply detects the act of placing waste in the trash can.

[0237] Step 2:

[0238] The terminal transmits the acquired image data to the server in real time. The images are converted to a standard format and programmed to prevent errors. This ensures that the analysis process can begin without delay.

[0239] Step 3:

[0240] The server inputs the received image data into an AI image recognition module to identify the type of waste. The AI ​​uses a multi-layer neural network to classify objects into categories such as "paper," "plastic," and "metal" based on their shape and structure. This model has been pre-trained on a vast dataset, resulting in high accuracy.

[0241] Step 4:

[0242] The terminal reads the RFID tags in the waste using an RFID reader. The information obtained from these tags includes product IDs and detailed manufacturing information, which is then transmitted to a server, allowing for accurate identification of the waste's characteristics.

[0243] Step 5:

[0244] The server integrates the collected image recognition data and RFID data, and performs analysis by querying it against information stored in the database. This analysis helps determine how frequently specific types of waste are being introduced and how much room there is for improvement, thereby generating suggestions for product improvements.

[0245] Step 6:

[0246] Users earn eco-points by properly classifying their waste. These points are managed through a smartphone application, and accumulated points can be used for discount coupons or product purchases, contributing to increased environmental awareness among users.

[0247] Step 7:

[0248] The server automatically generates improvement suggestions based on the analysis results and notifies designated manufacturers and stakeholders. These suggestions are sent via email or a web portal and used as feedback for product improvement.

[0249] (Example 1)

[0250] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0251] In modern society, vast amounts of waste are generated, and its proper management has a significant impact on the global environment. However, effective waste classification and improved recycling rates require active participation from users, as well as product improvement suggestions from manufacturers. Given this situation, there is a lack of systems to promote efficient waste management and recycling activities.

[0252] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0253] In this invention, the server includes a device having the function of acquiring image information and identifying organic matter based on the image information, a device having the function of classifying substances based on the identification results of the organic matter, and a device having the function of reading identification tags attached to substances and acquiring usage history information of the substances. This enables users to accurately classify waste without being aware of it, generates appropriate improvement suggestions for producers, and promotes the efficiency of waste management and recycling activities.

[0254] "Image information" refers to digital data that visually captures the appearance of a substance and is used for identification and classification.

[0255] "Organic matter" refers to substances that include materials derived from living organisms and their processed products, and are classified as recyclable resources.

[0256] An "identification tag" is a small identification device attached to an object that has the function of electronically storing information such as usage history.

[0257] "Usage history information" refers to a series of pieces of information about a substance from its manufacture to its use and disposal, and is data used to understand the product's lifecycle.

[0258] A "device" refers to a combination of hardware and software designed to perform a specific function.

[0259] A "producer" refers to a corporation or individual responsible for manufacturing or supplying a product, and is the party to whom suggestions regarding product improvements should be received.

[0260] This invention is a system for achieving efficient waste management through the cooperation of a terminal, a server, and a user. Specifically, the terminal is installed in a smart trash can and is equipped with a sensor that activates when waste is placed inside and a camera for capturing images. This camera acquires image information of the waste and transmits that data to the server in real time. Through this operation, the waste is digitized without the user having to be particularly aware of it.

[0261] The server functions as a device that executes machine learning algorithms using the received image information. For example, it applies known deep learning models such as ResNet and VGG to identify the type of waste and classify it into categories such as "plastic," "paper," and "metal" by comparing it with a database. Furthermore, the terminal uses an RFID reader to read the identification tags attached to the materials and obtain information on the material's usage history. This provides information for a detailed analysis of the product's lifecycle.

[0262] The collected identification and usage history information is integrated on a server, and improvement suggestions are generated for specific producers. These suggestions include specific improvement plans aimed at increasing the efficiency of waste recycling and are communicated to producers or relevant parties through a predetermined communication method. This facilitates improvements during the product development stage.

[0263] Users are rewarded for correctly classifying their waste, and these rewards can be managed through a dedicated application or web portal. Users can use these rewards to exchange for benefits within their local community.

[0264] As a concrete example, when a user places a plastic bottle into a smart trash can, the device takes a picture of it with its camera, and the image sent to the server is analyzed and classified as recyclable "plastic." The RFID tag attached to the plastic bottle is read, and details such as "Manufacturer: General Incorporated Association, Manufacturing Date: Specified Date" are obtained, and the product's recycling data is updated based on this information. An example of a prompt to be input into the generating AI model would be, "Explain how paper waste is identified in the automated waste classification system. Include the name of the relevant machine learning algorithm and processing procedure."

[0265] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0266] Step 1:

[0267] The device uses sensors to detect when waste is placed in the smart trash can. This detection triggers a camera to activate, taking an image of the waste and generating image data. The input is the physical placement of waste, and the output is digital image data. The generated image data is immediately sent to the server.

[0268] Step 2:

[0269] The server receives image data from the terminal. Next, it inputs the image data into a machine learning algorithm to identify the type of waste. This process utilizes generative AI models such as ResNet and VGG to analyze image features and classify the waste into categories such as "plastic," "paper," and "metal." The input is image data, and the output is the category information of the classified waste.

[0270] Step 3:

[0271] The terminal reads the identification tag attached to the waste using an RFID reader. This retrieves product lifecycle data, such as manufacturer and manufacturing date. The input is the RFID tag attached to the waste, and the output is the usage history information contained within the tag. This information is sent to a server for product tracking and analysis.

[0272] Step 4:

[0273] The server integrates the category information of classified waste with usage history information obtained from RFID readers. Based on this information, it performs analysis to identify the recycling rate of products and areas for improvement at a specific producer. Based on the analysis results, it automatically generates improvement suggestions. The input is integrated information, and the output is improvement suggestions for producers.

[0274] Step 5:

[0275] When a user correctly sorts waste, the server calculates and awards eco-points as a corresponding reward. These points are managed through a dedicated application or web portal, and users can use them to exchange for local benefits. Inputs are the sorting results and user ID, and output is the awarded eco-point information.

[0276] Through these steps, the system enables efficient waste management and supports active user participation and product improvement.

[0277] (Application Example 1)

[0278] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as a "terminal".

[0279] Although proper classification of waste is important in urban environments and corporate activities, it is cumbersome for individual users to consciously classify waste, resulting in incorrect disposal and a decrease in the recycling rate. In addition, since the motivation for users to promote recycling activities is insufficient, there is a demand for improving the efficiency of waste management. Furthermore, it is also an issue that feedback on product improvement to manufacturers is not carried out promptly.

[0280] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following respective means.

[0281] In this invention, the server includes an acquisition means for acquiring image information to identify waste, a classification means for classifying waste based on the identification result, a reading means for acquiring and analyzing life cycle information, and a presentation means for displaying waste classification information and eco-points on the user's device. Thereby, real-time classification of waste, effective utilization of the eco-point system, and product improvement by prompt feedback become possible.

[0282] The "acquisition means" is a device having a function of acquiring image information related to waste and performing identification based on the data.

[0283] The "classification means" is a device having a function of analyzing the acquired image information of waste and classifying it into appropriate categories.

[0284] The "reading means" is a device having a function of reading an identification tag attached to waste and acquiring its life cycle information.

[0285] The "analysis means" is a device that comprehensively analyzes the acquired life cycle information and identification information and has the function of generating proposals for product improvement for manufacturers.

[0286] The "display means" is a device that has the function of visually displaying waste classification information and the status of eco-points in real time on the user's device.

[0287] An "eco-point" is a point given to a user when waste is correctly classified and processed, and it is a system for incentivizing the user's environmental protection activities.

[0288] The system for implementing this invention consists of three components: a server, a terminal, and a user. For the entire system to function smoothly, these elements need to play their respective roles. First, the terminal uses sensors and cameras installed in smart trash cans to acquire image data when waste is thrown in. Specifically, this camera takes high-resolution images of the waste, and the RFID reader reads the identification tag attached to the waste. This information is immediately sent to the server.

[0289] The server uses image recognition libraries such as OpenCV and TensorFlow to process the received image data. Thereby, the types of waste are classified into "plastic", "paper", "metal", etc. Next, the server stores the read life cycle information in the database and performs analysis. The analysis results are used to evaluate the traceability and recyclability of products. Furthermore, the server grants eco-points to users who have correctly classified the waste based on the analysis data.

[0290] Users can check eco-points and waste classification information in real time using their smartphones or smart glasses. The system visually displays the classification results and suggestions for using eco-points on these devices. For example, if a user correctly recycles plastic, a message such as "Your plastic has been correctly recycled. Eco-points have been awarded." will be displayed.

[0291] As a concrete example, when a user disposes of waste at home, using smart glasses would provide real-time information such as, "This is metal. Recycling it will increase your eco-points." Such a system would create an incentive for users to participate more actively in recycling activities.

[0292] Furthermore, an example of a prompt message from the AI ​​model in this system is a text-based message such as, "Based on the photo and RFID information of this waste, explain how to classify it and calculate and present the appropriate eco-points." This incentivizes the server to perform appropriate classification and rewarding.

[0293] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0294] Step 1:

[0295] The terminal uses a camera mounted on the smart trash can to acquire image data when waste is placed inside. This acquired image becomes the input. The input image data is immediately transmitted from the terminal to the server for identification. As preparation for transmission, a specific operation is performed to convert the image data format.

[0296] Step 2:

[0297] The server identifies the type of waste from the received image data. This process uses image recognition libraries such as OpenCV or TensorFlow, with the image data as input. The server performs pattern analysis on the data and obtains output classified into categories such as "plastic," "metal," and "paper."

[0298] Step 3:

[0299] The terminal uses an RFID reader to read the identification tag attached to the waste. The reading result is sent to the server as input. Specifically, this involves decoding the tag information and saving it to a database.

[0300] Step 4:

[0301] The server performs analysis based on the lifecycle information and identification results. It receives lifecycle information and identification results as input, performs data analysis, and outputs an optimized recycling method. These results are used for future improvements and generate feedback reports for manufacturers.

[0302] Step 5:

[0303] The server calculates and awards eco-points for waste correctly classified by the user. The server uses the classification results as input, performs calculations to award eco-points, and outputs them as the user's score. Specifically, point records and account integration are performed.

[0304] Step 6:

[0305] Users can check waste classification information and eco-point status through an application on their device. Based on output data transmitted from the server, the device visualizes and presents the information to the user. Specifically, it performs real-time information updates and notifications.

[0306] Furthermore, an emotion engine for estimating the user's emotions may be combined. That is, the specific processing unit 290 may estimate the user's emotions using the emotion specific model 59 and perform specific processing using the user's emotions.

[0307] The present invention is a system that combines an acquisition means, a classification means, a reading means, an analysis means, and an emotion recognition engine in order to improve the efficiency of waste management. This system supports the user in appropriately classifying waste, analyzes the user's emotional state, and provides feedback and process adjustment.

[0308] First, the terminal incorporates a sensor into the smart trash can to detect when waste has been inserted. At this time, image data is acquired using a camera. As a result, a mechanism is provided in which data is automatically collected every time the user discards waste. This image data is transmitted to the server in real time to prepare for analysis.

[0309] The server processes the received image data with an AI image recognition module to identify the type of waste. The identified data is not only used for waste classification but also serves for overall analysis. Also, the terminal uses an RFID reader to obtain life cycle data from the identification tag attached to the waste and transmits this to the server.

[0310] Furthermore, the server uses an emotion recognition engine to analyze the user's facial expressions and voice tones obtained by the camera to identify the user's emotional state. Based on this emotion data, the system optimizes the interface and feedback content for the user. For example, if it is determined that the user is stressed about waste classification, guidance and assistance are provided to make the process more intuitive.

[0311] Furthermore, users who properly sort their waste will be awarded eco-points. These points serve as a means to raise users' environmental awareness and promote recycling activities. Eco-points are managed through a dedicated application and can be used for various purposes.

[0312] Ultimately, the server generates suggestions for product improvement and environmental impact reduction based on all identification and analysis data, and notifies the relevant manufacturers and responsible parties. This notification process is carried out quickly and efficiently via email and a web portal.

[0313] Thus, by implementing the present invention, it is expected that user convenience in waste management will be enhanced and that the realization of a sustainable society will be contributed to.

[0314] The following describes the processing flow.

[0315] Step 1:

[0316] The terminal uses sensors to detect when waste is dumped. When waste is discarded, the camera automatically activates and takes an image. This image records the appearance of the discarded waste. The captured image data is sent to a server for analysis.

[0317] Step 2:

[0318] The server passes the received image data to an AI image recognition module, which analyzes the type of waste. For example, it is classified into categories such as "paper," "plastic," and "metal." This classification information is used to support the proper disposal of the waste.

[0319] Step 3:

[0320] The terminal uses an RFID reader to read identification tags attached to waste. Manufacturer and product information is obtained from these tags and sent to a server as lifecycle data. This makes it possible to track the origin and characteristics of the waste.

[0321] Step 4:

[0322] The server analyzes the identification data and lifecycle information of the collected waste. Specifically, it performs trend analysis and automatically generates improvement suggestions for manufacturers. This provides concrete measures for manufacturers to reduce their environmental impact.

[0323] Step 5:

[0324] The server uses an emotion recognition engine to analyze the user's facial expressions and voice patterns to assess their emotional state. For example, if the user is stressed, the server adjusts the interface to simplify operations and improve the user experience.

[0325] Step 6:

[0326] Users are awarded eco-points for properly classifying their waste. These points are managed through a dedicated application, allowing for easy checking and use. Eco-points can be used for discounts on reusable products and other benefits, increasing users' contribution to the environment.

[0327] Step 7:

[0328] Based on the analysis and sentiment recognition results, the server notifies the relevant manufacturer or person in charge of product improvement suggestions. Notifications are made via email or a dedicated portal, creating a rapid feedback loop. This facilitates continuous product improvement.

[0329] (Example 2)

[0330] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0331] In waste management, it is necessary to efficiently identify and classify waste appropriately, provide feedback that takes user sentiment into consideration, and promote recycling activities. Furthermore, it is crucial to use the acquired data to drive product improvements in the manufacturing sector. Current systems suffer from insufficient waste identification and classification, as well as user support, leading to delays in suggestions and notifications to the manufacturing sector.

[0332] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0333] In this invention, the server includes an acquisition means having the function of acquiring image data and identifying waste based on the image data; a reading means having the function of reading information tags associated with the waste and acquiring waste usage history information; and an analysis means having the function of identifying the user's emotional state based on camera images and audio data and optimizing the interface and feedback content based on the emotional state. This enables appropriate identification and classification of waste, provides appropriate feedback according to the user's emotional state, and facilitates rapid improvement suggestions in the manufacturing field.

[0334] "Image data" refers to visual information acquired to represent the appearance of waste.

[0335] "Acquisition means" refers to a device or function for collecting image data.

[0336] "Classification means" refers to a device or function for dividing identified waste into specific categories.

[0337] "Reading means" refers to a device or function for detecting information tags associated with waste and collecting that information.

[0338] An "information tag" is an identifier attached to waste materials to record information about their usage history.

[0339] "Usage history information" refers to information about the waste's manufacturer, usage route, and recovery status.

[0340] "Analysis means" refers to a device or function for analyzing the condition of waste and users based on acquired data.

[0341] "Emotional state" refers to information that represents the user's emotions and is identified through facial recognition and voice analysis.

[0342] An "interface" is a mechanism or method for a user to interact with a system.

[0343] "Feedback" refers to information and instructions provided to a user, used to support their actions.

[0344] An "improvement suggestion" is advice or recommendation based on data analysis to encourage improvements to products and processes in the manufacturing sector.

[0345] "Network infrastructure" refers to the communication infrastructure used to transmit data.

[0346] This invention is a system for improving the efficiency of waste management, and includes acquisition means, classification means, reading means, analysis means, and feedback optimization function. When a user disposes of waste, this system acquires image data and audio data, and identifies and analyzes the type of waste based on that data. Furthermore, it analyzes the user's emotional state and provides feedback tailored to the user.

[0347] Specifically, the device incorporates sensors into the smart trash can to detect when waste is placed inside. These sensors include infrared and pressure sensors. Once waste is detected, a camera activates to capture high-resolution images of the waste. This image data is transmitted to a server in real time. The server uses a generative AI model to analyze the image data and identify the type of waste. For example, it may be classified as plastic, metal, or paper. This classification result is stored in a database and used to improve waste management.

[0348] Simultaneously, the terminal reads the information tags attached to the waste and obtains lifecycle data. This data, which includes the product's manufacturer and usage history, is transferred to a server and used for further analysis.

[0349] Furthermore, the server incorporates an emotion recognition engine that uses the camera and voice input capabilities to analyze the user's facial expressions and tone of voice. The emotion data obtained from this analysis is used to optimize the user interface. If the user shows difficulty with waste classification, the system provides assistance and offers instructions to improve the classification process.

[0350] Furthermore, an eco-point system has been introduced, where users are awarded points for correctly classifying their waste. These points not only raise users' environmental awareness but can also be exchanged for rewards. Points are managed using a dedicated application and can be exchanged for things like discount coupons at supermarkets.

[0351] Ultimately, the server generates improvement suggestions based on all identification and sentiment data and notifies managers in the relevant manufacturing sectors. This notification is made via email or a web portal to enable rapid feedback. For example, a prompt for the generating AI model might say, "Suggest ways to optimize the recycling of plastic bottles using a waste management system."

[0352] Thus, this invention improves the efficiency of waste management while simultaneously personalizing the user experience and contributing to the realization of a sustainable society.

[0353] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0354] Step 1:

[0355] The device detects waste being placed in the smart trash can using a sensor built into the trash can. This sensor uses infrared technology and emits a signal when waste passes through the opening. The input is a physical change recognized by the sensor. By detecting this change, an output is obtained that recognizes that waste has been placed in the trash can.

[0356] Step 2:

[0357] The device activates its camera and acquires image data of the waste. Multiple images are taken from different angles and stored as high-resolution visual information. The input is the physical shape and color of the waste, and the output is image data. This data is transmitted to the server in real time.

[0358] Step 3:

[0359] The server analyzes the received image data using a generation AI model. The input is the transmitted image data, and the AI ​​identifies the type of waste based on a deep learning algorithm. This analysis generates waste category information as output. For example, it might be classified as plastic, metal, paper, etc.

[0360] Step 4:

[0361] The terminal reads information tags attached to waste and acquires lifecycle data using radio frequency identification (RFID) technology. The input is the RFID tag, and the output is sent to the server as usage history information. This information provides detailed data regarding the origin and usage route of the waste.

[0362] Step 5:

[0363] The server uses an emotion recognition engine to analyze the user's facial expressions and voice tone captured by the camera. Input includes the user's facial image and voice data. The output identifies the user's emotional state, and the system uses this data to provide corresponding feedback.

[0364] Step 6:

[0365] Based on the user's emotional state, the system optimizes feedback and the interface. Specifically, if it determines that the user is experiencing stress, it displays guidelines to simplify operation or provides voice assistance. The input is the user's emotional data, and the output is optimized user interface information.

[0366] Step 7:

[0367] The system awards eco-points to users who correctly classify their waste. Input includes information about the accuracy of the classification, and the points are recorded in the user's account as output. These points can be managed and viewed through a dedicated application.

[0368] Step 8:

[0369] The server analyzes all collected data and generates improvement suggestions for the manufacturing sector. Inputs include identification results and usage history information, while output lists specific actions as improvement suggestions. These suggestions are then communicated to relevant parties via email and a web portal.

[0370] (Application Example 2)

[0371] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0372] In today's mass-consumption society, proper waste classification and recycling are crucial issues from an environmental protection perspective. However, users often struggle to accurately classify waste, and the process can be stressful. Therefore, there is a need for support systems to improve the efficiency of waste management.

[0373] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0374] In this invention, the server includes an acquisition means having the function of acquiring image information and identifying waste based on said image information; a classification means having the function of classifying the waste based on the identification result of said waste; and an analysis means having the function of analyzing the user's emotional state and optimizing the user interface based on that data. This makes it easier for the user to intuitively classify waste and to provide appropriate feedback to further increase motivation.

[0375] "Image information" refers to visual data acquired through cameras and other imaging devices.

[0376] "Waste" refers to used products and unwanted materials that require proper disposal or recycling.

[0377] "Acquisition means" refers to a function for collecting image information of waste and using it within the system.

[0378] A "classification method" is a function that identifies the type of waste based on the image information acquired and sorts it into the appropriate category.

[0379] The "reading means" refers to a function that scans identification tags attached to waste materials to obtain their lifecycle information.

[0380] "Analysis means" refers to a function that processes data based on lifecycle information and identification results, and generates suggestions for improving the user interface.

[0381] "Emotional state" refers to the psychological state analyzed from the user's facial expressions, voice, etc., and is an element that the system uses to optimize interaction.

[0382] An "interface" is a user interaction function that allows users to interact with a system and exchange information.

[0383] "Eco-points" are rewards given to users who properly classify their waste, and serve as an incentive to promote environmental awareness.

[0384] In the system implementing this invention, a camera installed on a terminal first collects image information of the waste. The acquired image information is sent to a server, where an AI image recognition model is used to identify the type of waste. Specific software used includes PyTorch and TensorFlow. Based on this identification data, the server appropriately classifies the waste.

[0385] The terminal also features an RFID reader to read identification tags attached to waste items, acquiring lifecycle information. The acquired lifecycle information and identification data are analyzed on a server. During this process, since the user's emotional state affects the user interface, an emotion recognition engine is used to analyze the user's facial expressions and voice to identify their state. Based on the analysis results, support information and feedback are provided to help the user process waste more easily.

[0386] Furthermore, users who correctly classify their waste are awarded eco-points, which are managed through a dedicated application. This point system serves as an incentive to raise users' environmental awareness. The server also has the ability to generate suggestions for product improvement and environmental impact reduction based on all identification and analysis data, and notify relevant organizations. These notifications are sent via email and a web portal.

[0387] A concrete example is a system installed at a recycling center in a certain city. When a user disposes of a plastic bottle and records the process through smart glasses, the bottle is immediately identified and the interface is adjusted accordingly. If the user shows signs of confusion based on sentiment analysis, the system provides simple guidance. Furthermore, if the disposal is done properly, eco-points are added to the user's smartphone app, encouraging community use.

[0388] Examples of prompts for a generative AI model:

[0389] "Please explain, with examples, how a smart recycling assistant app can improve residents' waste disposal habits."

[0390] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0391] Step 1:

[0392] The device uses a sensor to detect when waste is placed in a trash can and a camera to capture an image of the waste. This image is immediately sent to a server and processed as image information. The input is image data from the camera, and the output is digital data containing that information.

[0393] Step 2:

[0394] The server analyzes the received image information using an AI image recognition module to identify the type of waste. This process inputs the acquired image data into an AI model and outputs an identification label as a classification result. Specifically, an identification algorithm using TensorFlow is in operation.

[0395] Step 3:

[0396] Upon receiving the identification result, the terminal uses an RFID reader to read the identification tag attached to the waste. It then obtains the waste's lifecycle information from this tag and transmits it to the server. The input is the signal from the identification tag, and the output is the lifecycle information derived from that data.

[0397] Step 4:

[0398] The server activates an emotion recognition engine based on lifecycle information and identification results, and analyzes the user's emotional state using camera and audio data. The input is the user's facial expressions and audio data, and the output is an evaluation of their emotional state. OpenCV and the emotion_recognition library are used in this process.

[0399] Step 5:

[0400] Based on the analyzed data, the server optimizes the user interface and provides support information in real time. If the user is experiencing stress, it presents more user-friendly guides and assistance. The input is emotional evaluation, and the output is an optimized interface and feedback.

[0401] Step 6:

[0402] If a user correctly classifies their waste, the server calculates eco-points and adds them to the user's account. This process calculates the number of points based on the classification results and sends that information to the user's application. The input is the accuracy of the classification, and the output is a notification of point addition.

[0403] Step 7:

[0404] The server integrates all the data and generates suggestions for product improvements and reducing environmental impact. These suggestions are communicated to relevant organizations via email and a web portal. The input is the entire analyzed data, and the output is the suggestion notification message.

[0405] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0406] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0407] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0408] [Third Embodiment]

[0409] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0410] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0411] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0412] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0413] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0414] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0415] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0416] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0417] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0418] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0419] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0420] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0421] This invention is an automated classification system aimed at improving the efficiency of waste management, and mainly consists of acquisition means, classification means, reading means, and analysis means. This system assists users, such as households and businesses, in properly classifying waste.

[0422] First, the device incorporates sensors into the smart trash can and captures image data with a camera when waste is placed inside. This means that waste information is digitized without the user's awareness each time they throw away trash. The images captured by the camera are immediately sent to a server for analysis.

[0423] Next, the server processes the received image data and uses its built-in machine learning algorithms to identify the type of waste. This process allows for classification into categories such as "plastic," "paper," and "metal." The identified data contributes to further analysis of waste trends and recycling potential.

[0424] The terminal also reads identification tags attached to waste using an RFID reader to obtain product lifecycle information. This information includes details such as the manufacturing date and manufacturer, which helps in product tracking and improving the recycling process.

[0425] The server then integrates and analyzes the collected identification information and lifecycle data. This analysis makes it possible to determine the extent to which specific products are recycled and to identify products that need improvement. Based on the analysis results, the system automatically generates a report suggesting product improvements for specific manufacturers.

[0426] Furthermore, the system awards eco-points to users who correctly classify their waste. These eco-points are managed through a separate application or web portal, and users can use them for various purposes. This system motivates users to actively classify their waste correctly and contribute to recycling efforts.

[0427] It is expected that the implementation of this invention will lead to efficient waste management and improved recycling rates, thereby contributing to the formation of a sustainable society.

[0428] The following describes the processing flow.

[0429] Step 1:

[0430] The device uses sensors to detect when waste is placed in the smart trash can. When the sensor is triggered, the camera automatically activates and takes a picture of the waste. At this stage, the sensor does not detect the type or amount of waste; it simply detects the act of placing waste in the trash can.

[0431] Step 2:

[0432] The terminal transmits the acquired image data to the server in real time. The images are converted to a standard format and programmed to prevent errors. This ensures that the analysis process can begin without delay.

[0433] Step 3:

[0434] The server inputs the received image data into an AI image recognition module to identify the type of waste. The AI ​​uses a multi-layer neural network to classify objects into categories such as "paper," "plastic," and "metal" based on their shape and structure. This model has been pre-trained on a vast dataset, resulting in high accuracy.

[0435] Step 4:

[0436] The terminal reads the RFID tags in the waste using an RFID reader. The information obtained from these tags includes product IDs and detailed manufacturing information, which is then transmitted to a server, allowing for accurate identification of the waste's characteristics.

[0437] Step 5:

[0438] The server integrates the collected image recognition data and RFID data, and performs analysis by querying it against information stored in the database. This analysis helps determine how frequently specific types of waste are being introduced and how much room there is for improvement, thereby generating suggestions for product improvements.

[0439] Step 6:

[0440] Users earn eco-points by properly classifying their waste. These points are managed through a smartphone application, and accumulated points can be used for discount coupons or product purchases, contributing to increased environmental awareness among users.

[0441] Step 7:

[0442] The server automatically generates improvement suggestions based on the analysis results and notifies designated manufacturers and stakeholders. These suggestions are sent via email or a web portal and used as feedback for product improvement.

[0443] (Example 1)

[0444] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0445] In modern society, vast amounts of waste are generated, and its proper management has a significant impact on the global environment. However, effective waste classification and improved recycling rates require active participation from users, as well as product improvement suggestions from manufacturers. Given this situation, there is a lack of systems to promote efficient waste management and recycling activities.

[0446] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0447] In this invention, the server includes a device having the function of acquiring image information and identifying organic matter based on the image information, a device having the function of classifying substances based on the identification results of the organic matter, and a device having the function of reading identification tags attached to substances and acquiring usage history information of the substances. This enables users to accurately classify waste without being aware of it, generates appropriate improvement suggestions for producers, and promotes the efficiency of waste management and recycling activities.

[0448] "Image information" refers to digital data that visually captures the appearance of a substance and is used for identification and classification.

[0449] "Organic matter" refers to substances that include materials derived from living organisms and their processed products, and are classified as recyclable resources.

[0450] An "identification tag" is a small identification device attached to an object that has the function of electronically storing information such as usage history.

[0451] "Usage history information" refers to a series of pieces of information about a substance from its manufacture to its use and disposal, and is data used to understand the product's lifecycle.

[0452] A "device" refers to a combination of hardware and software designed to perform a specific function.

[0453] A "producer" refers to a corporation or individual responsible for manufacturing or supplying a product, and is the party to whom suggestions regarding product improvements should be received.

[0454] This invention is a system for achieving efficient waste management through the cooperation of a terminal, a server, and a user. Specifically, the terminal is installed in a smart trash can and is equipped with a sensor that activates when waste is placed inside and a camera for capturing images. This camera acquires image information of the waste and transmits that data to the server in real time. Through this operation, the waste is digitized without the user having to be particularly aware of it.

[0455] The server functions as a device that executes machine learning algorithms using the received image information. For example, it applies known deep learning models such as ResNet and VGG to identify the type of waste and classify it into categories such as "plastic," "paper," and "metal" by comparing it with a database. Furthermore, the terminal uses an RFID reader to read the identification tags attached to the materials and obtain information on the material's usage history. This provides information for a detailed analysis of the product's lifecycle.

[0456] The collected identification and usage history information is integrated on a server, and improvement suggestions are generated for specific producers. These suggestions include specific improvement plans aimed at increasing the efficiency of waste recycling and are communicated to producers or relevant parties through a predetermined communication method. This facilitates improvements during the product development stage.

[0457] Users are rewarded for correctly classifying their waste, and these rewards can be managed through a dedicated application or web portal. Users can use these rewards to exchange for benefits within their local community.

[0458] As a concrete example, when a user places a plastic bottle into a smart trash can, the device takes a picture of it with its camera, and the image sent to the server is analyzed and classified as recyclable "plastic." The RFID tag attached to the plastic bottle is read, and details such as "Manufacturer: General Incorporated Association, Manufacturing Date: Specified Date" are obtained, and the product's recycling data is updated based on this information. An example of a prompt to be input into the generating AI model would be, "Explain how paper waste is identified in the automated waste classification system. Include the name of the relevant machine learning algorithm and processing procedure."

[0459] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0460] Step 1:

[0461] The device uses sensors to detect when waste is placed in the smart trash can. This detection triggers a camera to activate, taking an image of the waste and generating image data. The input is the physical placement of waste, and the output is digital image data. The generated image data is immediately sent to the server.

[0462] Step 2:

[0463] The server receives image data from the terminal. Next, it inputs the image data into a machine learning algorithm to identify the type of waste. This process utilizes generative AI models such as ResNet and VGG to analyze image features and classify the waste into categories such as "plastic," "paper," and "metal." The input is image data, and the output is the category information of the classified waste.

[0464] Step 3:

[0465] The terminal reads the identification tag attached to the waste using an RFID reader. This retrieves product lifecycle data, such as manufacturer and manufacturing date. The input is the RFID tag attached to the waste, and the output is the usage history information contained within the tag. This information is sent to a server for product tracking and analysis.

[0466] Step 4:

[0467] The server integrates the category information of classified waste with usage history information obtained from RFID readers. Based on this information, it performs analysis to identify the recycling rate of products and areas for improvement at a specific producer. Based on the analysis results, it automatically generates improvement suggestions. The input is integrated information, and the output is improvement suggestions for producers.

[0468] Step 5:

[0469] When a user correctly sorts waste, the server calculates and awards eco-points as a corresponding reward. These points are managed through a dedicated application or web portal, and users can use them to exchange for local benefits. Inputs are the sorting results and user ID, and output is the awarded eco-point information.

[0470] Through these steps, the system enables efficient waste management and supports active user participation and product improvement.

[0471] (Application Example 1)

[0472] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0473] Proper waste classification is crucial for urban environments and business operations, but the cumbersome process of individual users consciously classifying waste leads to misdisposal and low recycling rates. Furthermore, insufficient user motivation to promote recycling activities necessitates more efficient waste management. Another challenge is the lack of timely feedback to manufacturers regarding product improvements.

[0474] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0475] In this invention, the server includes an acquisition means for acquiring image information and identifying waste, a classification means for classifying waste based on the identification results, a reading means for acquiring and analyzing lifecycle information, and a presentation means for displaying waste classification information and eco-points on the user's device. This enables real-time waste classification, effective utilization of an eco-point system, and product improvement through rapid feedback.

[0476] "Acquisition means" refers to a device that has the function of acquiring image information related to waste and performing identification based on that data.

[0477] A "classification device" is a device that analyzes acquired image information of waste and classifies it into the appropriate category.

[0478] A "reading device" is a device that has the function of reading identification tags attached to waste and obtaining its lifecycle information.

[0479] The "analysis means" is a device that performs an analysis by comprehensively analyzing the acquired lifecycle information and identification information, and has the function of generating product improvement suggestions for manufacturers.

[0480] A "presentation means" is a device that has the function of visually displaying waste classification information and eco-point status in real time on the user's device.

[0481] "Eco-points" are points awarded to users when they properly classify and dispose of waste, and the system is designed to incentivize users' environmental protection activities.

[0482] The system implementing this invention consists of three components: a server, a terminal, and a user. For the entire system to function smoothly, each of these components must fulfill its respective role. First, the terminal uses sensors and cameras installed in the smart trash can to acquire image data when waste is placed inside. Specifically, the camera takes high-resolution images of the waste, and the RFID reader reads the identification tag attached to the waste. This information is immediately transmitted to the server.

[0483] The server uses image recognition libraries such as OpenCV and TensorFlow to process the received image data. This classifies the waste into categories such as "plastic," "paper," and "metal." Next, the server stores the read lifecycle information in a database and performs analysis. The analysis results are used to evaluate the traceability and recyclability of the product. Furthermore, based on the analysis data, the server awards eco-points to users who correctly classify their waste.

[0484] Users can check eco-points and waste classification information in real time using their smartphones or smart glasses. The system visually displays the classification results and suggestions for using eco-points on these devices. For example, if a user correctly recycles plastic, a message such as "Your plastic has been correctly recycled. Eco-points have been awarded." will be displayed.

[0485] As a concrete example, when a user disposes of waste at home, using smart glasses would provide real-time information such as, "This is metal. Recycling it will increase your eco-points." Such a system would create an incentive for users to participate more actively in recycling activities.

[0486] Furthermore, an example of a prompt message from the AI ​​model in this system is a text-based message such as, "Based on the photo and RFID information of this waste, explain how to classify it and calculate and present the appropriate eco-points." This incentivizes the server to perform appropriate classification and rewarding.

[0487] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0488] Step 1:

[0489] The terminal uses a camera mounted on the smart trash can to acquire image data when waste is placed inside. This acquired image becomes the input. The input image data is immediately transmitted from the terminal to the server for identification. As preparation for transmission, a specific operation is performed to convert the image data format.

[0490] Step 2:

[0491] The server identifies the type of waste from the received image data. This process uses image recognition libraries such as OpenCV or TensorFlow, with the image data as input. The server performs pattern analysis on the data and obtains output classified into categories such as "plastic," "metal," and "paper."

[0492] Step 3:

[0493] The terminal uses an RFID reader to read the identification tag attached to the waste. The reading result is sent to the server as input. Specifically, this involves decoding the tag information and saving it to a database.

[0494] Step 4:

[0495] The server performs analysis based on the lifecycle information and identification results. It receives lifecycle information and identification results as input, performs data analysis, and outputs an optimized recycling method. These results are used for future improvements and generate feedback reports for manufacturers.

[0496] Step 5:

[0497] The server calculates and awards eco-points for waste correctly classified by the user. The server uses the classification results as input, performs calculations to award eco-points, and outputs them as the user's score. Specifically, point records and account integration are performed.

[0498] Step 6:

[0499] Users can check waste classification information and eco-point status through an application on their device. Based on output data transmitted from the server, the device visualizes and presents the information to the user. Specifically, it performs real-time information updates and notifications.

[0500] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0501] This invention is a system that combines acquisition means, classification means, reading means, analysis means, and emotion recognition engine to improve the efficiency of waste management. This system assists the user in properly classifying waste while simultaneously analyzing the user's emotional state and providing feedback and adjusting the process.

[0502] First, the device incorporates sensors into the smart trash can to detect when waste is placed inside. At this time, a camera is used to acquire image data. This system automatically collects data every time the user disposes of waste. This image data is transmitted to a server in real time, ready for analysis.

[0503] The server processes the received image data using an AI image recognition module to identify the type of waste. The identified data is used not only for waste classification but also for overall analysis. The terminal also uses an RFID reader to obtain lifecycle data from identification tags attached to the waste and transmits this data to the server.

[0504] Furthermore, the server uses an emotion recognition engine to analyze the user's facial expressions and voice tone captured by the camera to identify the user's emotional state. Based on this emotional data, the system optimizes the interface and feedback provided to the user. For example, if it determines that the user is experiencing stress during waste sorting, it provides guidance and assistance to make the process more intuitive.

[0505] Furthermore, users who properly sort their waste will be awarded eco-points. These points serve as a means to raise users' environmental awareness and promote recycling activities. Eco-points are managed through a dedicated application and can be used for various purposes.

[0506] Ultimately, the server generates suggestions for product improvement and environmental impact reduction based on all identification and analysis data, and notifies the relevant manufacturers and responsible parties. This notification process is carried out quickly and efficiently via email and a web portal.

[0507] Thus, by implementing the present invention, it is expected that user convenience in waste management will be enhanced and that the realization of a sustainable society will be contributed to.

[0508] The following describes the processing flow.

[0509] Step 1:

[0510] The terminal uses sensors to detect when waste is dumped. When waste is discarded, the camera automatically activates and takes an image. This image records the appearance of the discarded waste. The captured image data is sent to a server for analysis.

[0511] Step 2:

[0512] The server passes the received image data to an AI image recognition module, which analyzes the type of waste. For example, it is classified into categories such as "paper," "plastic," and "metal." This classification information is used to support the proper disposal of the waste.

[0513] Step 3:

[0514] The terminal uses an RFID reader to read identification tags attached to waste. Manufacturer and product information is obtained from these tags and sent to a server as lifecycle data. This makes it possible to track the origin and characteristics of the waste.

[0515] Step 4:

[0516] The server analyzes the identification data and lifecycle information of the collected waste. Specifically, it performs trend analysis and automatically generates improvement suggestions for manufacturers. This provides concrete measures for manufacturers to reduce their environmental impact.

[0517] Step 5:

[0518] The server uses an emotion recognition engine to analyze the user's facial expressions and voice patterns to assess their emotional state. For example, if the user is stressed, the server adjusts the interface to simplify operations and improve the user experience.

[0519] Step 6:

[0520] Users are awarded eco-points for properly classifying their waste. These points are managed through a dedicated application, allowing for easy checking and use. Eco-points can be used for discounts on reusable products and other benefits, increasing users' contribution to the environment.

[0521] Step 7:

[0522] Based on the analysis and sentiment recognition results, the server notifies the relevant manufacturer or person in charge of product improvement suggestions. Notifications are made via email or a dedicated portal, creating a rapid feedback loop. This facilitates continuous product improvement.

[0523] (Example 2)

[0524] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0525] In waste management, it is necessary to efficiently identify and classify waste appropriately, provide feedback that takes user sentiment into consideration, and promote recycling activities. Furthermore, it is crucial to use the acquired data to drive product improvements in the manufacturing sector. Current systems suffer from insufficient waste identification and classification, as well as user support, leading to delays in suggestions and notifications to the manufacturing sector.

[0526] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0527] In this invention, the server includes an acquisition means having the function of acquiring image data and identifying waste based on the image data; a reading means having the function of reading information tags associated with the waste and acquiring waste usage history information; and an analysis means having the function of identifying the user's emotional state based on camera images and audio data and optimizing the interface and feedback content based on the emotional state. This enables appropriate identification and classification of waste, provides appropriate feedback according to the user's emotional state, and facilitates rapid improvement suggestions in the manufacturing field.

[0528] "Image data" refers to visual information acquired to represent the appearance of waste.

[0529] "Acquisition means" refers to a device or function for collecting image data.

[0530] "Classification means" refers to a device or function for dividing identified waste into specific categories.

[0531] "Reading means" refers to a device or function for detecting information tags associated with waste and collecting that information.

[0532] An "information tag" is an identifier attached to waste materials to record information about their usage history.

[0533] "Usage history information" refers to information about the waste's manufacturer, usage route, and recovery status.

[0534] "Analysis means" refers to a device or function for analyzing the condition of waste and users based on acquired data.

[0535] "Emotional state" refers to information that represents the user's emotions and is identified through facial recognition and voice analysis.

[0536] An "interface" is a mechanism or method for a user to interact with a system.

[0537] "Feedback" refers to information and instructions provided to a user, used to support their actions.

[0538] An "improvement suggestion" is advice or recommendation based on data analysis to encourage improvements to products and processes in the manufacturing sector.

[0539] "Network infrastructure" refers to the communication infrastructure used to transmit data.

[0540] This invention is a system for improving the efficiency of waste management, and includes acquisition means, classification means, reading means, analysis means, and feedback optimization function. When a user disposes of waste, this system acquires image data and audio data, and identifies and analyzes the type of waste based on that data. Furthermore, it analyzes the user's emotional state and provides feedback tailored to the user.

[0541] Specifically, the device incorporates sensors into the smart trash can to detect when waste is placed inside. These sensors include infrared and pressure sensors. Once waste is detected, a camera activates to capture high-resolution images of the waste. This image data is transmitted to a server in real time. The server uses a generative AI model to analyze the image data and identify the type of waste. For example, it may be classified as plastic, metal, or paper. This classification result is stored in a database and used to improve waste management.

[0542] Simultaneously, the terminal reads the information tags attached to the waste and obtains lifecycle data. This data, which includes the product's manufacturer and usage history, is transferred to a server and used for further analysis.

[0543] Furthermore, the server incorporates an emotion recognition engine that uses the camera and voice input capabilities to analyze the user's facial expressions and tone of voice. The emotion data obtained from this analysis is used to optimize the user interface. If the user shows difficulty with waste classification, the system provides assistance and offers instructions to improve the classification process.

[0544] Furthermore, an eco-point system has been introduced, where users are awarded points for correctly classifying their waste. These points not only raise users' environmental awareness but can also be exchanged for rewards. Points are managed using a dedicated application and can be exchanged for things like discount coupons at supermarkets.

[0545] Ultimately, the server generates improvement suggestions based on all identification and sentiment data and notifies managers in the relevant manufacturing sectors. This notification is made via email or a web portal to enable rapid feedback. For example, a prompt for the generating AI model might say, "Suggest ways to optimize the recycling of plastic bottles using a waste management system."

[0546] Thus, this invention improves the efficiency of waste management while simultaneously personalizing the user experience and contributing to the realization of a sustainable society.

[0547] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0548] Step 1:

[0549] The device detects waste being placed in the smart trash can using a sensor built into the trash can. This sensor uses infrared technology and emits a signal when waste passes through the opening. The input is a physical change recognized by the sensor. By detecting this change, an output is obtained that recognizes that waste has been placed in the trash can.

[0550] Step 2:

[0551] The device activates its camera and acquires image data of the waste. Multiple images are taken from different angles and stored as high-resolution visual information. The input is the physical shape and color of the waste, and the output is image data. This data is transmitted to the server in real time.

[0552] Step 3:

[0553] The server analyzes the received image data using a generation AI model. The input is the transmitted image data, and the AI ​​identifies the type of waste based on a deep learning algorithm. This analysis generates waste category information as output. For example, it might be classified as plastic, metal, paper, etc.

[0554] Step 4:

[0555] The terminal reads information tags attached to waste and acquires lifecycle data using radio frequency identification (RFID) technology. The input is the RFID tag, and the output is sent to the server as usage history information. This information provides detailed data regarding the origin and usage route of the waste.

[0556] Step 5:

[0557] The server uses an emotion recognition engine to analyze the user's facial expressions and voice tone captured by the camera. Input includes the user's facial image and voice data. The output identifies the user's emotional state, and the system uses this data to provide corresponding feedback.

[0558] Step 6:

[0559] Based on the user's emotional state, the system optimizes feedback and the interface. Specifically, if it determines that the user is experiencing stress, it displays guidelines to simplify operation or provides voice assistance. The input is the user's emotional data, and the output is optimized user interface information.

[0560] Step 7:

[0561] The system awards eco-points to users who correctly classify their waste. Input includes information about the accuracy of the classification, and the points are recorded in the user's account as output. These points can be managed and viewed through a dedicated application.

[0562] Step 8:

[0563] The server analyzes all collected data and generates improvement suggestions for the manufacturing sector. Inputs include identification results and usage history information, while output lists specific actions as improvement suggestions. These suggestions are then communicated to relevant parties via email and a web portal.

[0564] (Application Example 2)

[0565] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0566] In today's mass-consumption society, proper waste classification and recycling are crucial issues from an environmental protection perspective. However, users often struggle to accurately classify waste, and the process can be stressful. Therefore, there is a need for support systems to improve the efficiency of waste management.

[0567] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0568] In this invention, the server includes an acquisition means having the function of acquiring image information and identifying waste based on said image information; a classification means having the function of classifying the waste based on the identification result of said waste; and an analysis means having the function of analyzing the user's emotional state and optimizing the user interface based on that data. This makes it easier for the user to intuitively classify waste and to provide appropriate feedback to further increase motivation.

[0569] "Image information" refers to visual data acquired through cameras and other imaging devices.

[0570] "Waste" refers to used products and unwanted materials that require proper disposal or recycling.

[0571] "Acquisition means" refers to a function for collecting image information of waste and using it within the system.

[0572] A "classification method" is a function that identifies the type of waste based on the image information acquired and sorts it into the appropriate category.

[0573] The "reading means" refers to a function that scans identification tags attached to waste materials to obtain their lifecycle information.

[0574] "Analysis means" refers to a function that processes data based on lifecycle information and identification results, and generates suggestions for improving the user interface.

[0575] "Emotional state" refers to the psychological state analyzed from the user's facial expressions, voice, etc., and is an element that the system uses to optimize interaction.

[0576] An "interface" is a user interaction function that allows users to interact with a system and exchange information.

[0577] "Eco-points" are rewards given to users who properly classify their waste, and serve as an incentive to promote environmental awareness.

[0578] In the system implementing this invention, a camera installed on a terminal first collects image information of the waste. The acquired image information is sent to a server, where an AI image recognition model is used to identify the type of waste. Specific software used includes PyTorch and TensorFlow. Based on this identification data, the server appropriately classifies the waste.

[0579] The terminal also features an RFID reader to read identification tags attached to waste items, acquiring lifecycle information. The acquired lifecycle information and identification data are analyzed on a server. During this process, since the user's emotional state affects the user interface, an emotion recognition engine is used to analyze the user's facial expressions and voice to identify their state. Based on the analysis results, support information and feedback are provided to help the user process waste more easily.

[0580] Furthermore, users who correctly classify their waste are awarded eco-points, which are managed through a dedicated application. This point system serves as an incentive to raise users' environmental awareness. The server also has the ability to generate suggestions for product improvement and environmental impact reduction based on all identification and analysis data, and notify relevant organizations. These notifications are sent via email and a web portal.

[0581] A concrete example is a system installed at a recycling center in a certain city. When a user disposes of a plastic bottle and records the process through smart glasses, the bottle is immediately identified and the interface is adjusted accordingly. If the user shows signs of confusion based on sentiment analysis, the system provides simple guidance. Furthermore, if the disposal is done properly, eco-points are added to the user's smartphone app, encouraging community use.

[0582] Examples of prompts for a generative AI model:

[0583] "Please explain, with examples, how a smart recycling assistant app can improve residents' waste disposal habits."

[0584] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0585] Step 1:

[0586] The device uses a sensor to detect when waste is placed in a trash can and a camera to capture an image of the waste. This image is immediately sent to a server and processed as image information. The input is image data from the camera, and the output is digital data containing that information.

[0587] Step 2:

[0588] The server analyzes the received image information using an AI image recognition module to identify the type of waste. This process inputs the acquired image data into an AI model and outputs an identification label as a classification result. Specifically, an identification algorithm using TensorFlow is in operation.

[0589] Step 3:

[0590] Upon receiving the identification result, the terminal uses an RFID reader to read the identification tag attached to the waste. It then obtains the waste's lifecycle information from this tag and transmits it to the server. The input is the signal from the identification tag, and the output is the lifecycle information derived from that data.

[0591] Step 4:

[0592] The server activates an emotion recognition engine based on lifecycle information and identification results, and analyzes the user's emotional state using camera and audio data. The input is the user's facial expressions and audio data, and the output is an evaluation of their emotional state. OpenCV and the emotion_recognition library are used in this process.

[0593] Step 5:

[0594] Based on the analyzed data, the server optimizes the user interface and provides support information in real time. If the user is experiencing stress, it presents more user-friendly guides and assistance. The input is emotional evaluation, and the output is an optimized interface and feedback.

[0595] Step 6:

[0596] If a user correctly classifies their waste, the server calculates eco-points and adds them to the user's account. This process calculates the number of points based on the classification results and sends that information to the user's application. The input is the accuracy of the classification, and the output is a notification of point addition.

[0597] Step 7:

[0598] The server integrates all the data and generates suggestions for product improvements and reducing environmental impact. These suggestions are communicated to relevant organizations via email and a web portal. The input is the entire analyzed data, and the output is the suggestion notification message.

[0599] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0600] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0601] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0602] [Fourth Embodiment]

[0603] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0604] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0605] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0606] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0607] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0608] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0609] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0610] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0611] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0612] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0613] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0614] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0615] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0616] This invention is an automated classification system aimed at improving the efficiency of waste management, and mainly consists of acquisition means, classification means, reading means, and analysis means. This system assists users, such as households and businesses, in properly classifying waste.

[0617] First, the device incorporates sensors into the smart trash can and captures image data with a camera when waste is placed inside. This means that waste information is digitized without the user's awareness each time they throw away trash. The images captured by the camera are immediately sent to a server for analysis.

[0618] Next, the server processes the received image data and uses its built-in machine learning algorithms to identify the type of waste. This process allows for classification into categories such as "plastic," "paper," and "metal." The identified data contributes to further analysis of waste trends and recycling potential.

[0619] The terminal also reads identification tags attached to waste using an RFID reader to obtain product lifecycle information. This information includes details such as the manufacturing date and manufacturer, which helps in product tracking and improving the recycling process.

[0620] The server then integrates and analyzes the collected identification information and lifecycle data. This analysis makes it possible to determine the extent to which specific products are recycled and to identify products that need improvement. Based on the analysis results, the system automatically generates a report suggesting product improvements for specific manufacturers.

[0621] Furthermore, the system awards eco-points to users who correctly classify their waste. These eco-points are managed through a separate application or web portal, and users can use them for various purposes. This system motivates users to actively classify their waste correctly and contribute to recycling efforts.

[0622] It is expected that the implementation of this invention will lead to efficient waste management and improved recycling rates, thereby contributing to the formation of a sustainable society.

[0623] The following describes the processing flow.

[0624] Step 1:

[0625] The device uses sensors to detect when waste is placed in the smart trash can. When the sensor is triggered, the camera automatically activates and takes a picture of the waste. At this stage, the sensor does not detect the type or amount of waste; it simply detects the act of placing waste in the trash can.

[0626] Step 2:

[0627] The terminal transmits the acquired image data to the server in real time. The images are converted to a standard format and programmed to prevent errors. This ensures that the analysis process can begin without delay.

[0628] Step 3:

[0629] The server inputs the received image data into an AI image recognition module to identify the type of waste. The AI ​​uses a multi-layer neural network to classify objects into categories such as "paper," "plastic," and "metal" based on their shape and structure. This model has been pre-trained on a vast dataset, resulting in high accuracy.

[0630] Step 4:

[0631] The terminal reads the RFID tags in the waste using an RFID reader. The information obtained from these tags includes product IDs and detailed manufacturing information, which is then transmitted to a server, allowing for accurate identification of the waste's characteristics.

[0632] Step 5:

[0633] The server integrates the collected image recognition data and RFID data, and performs analysis by querying it against information stored in the database. This analysis helps determine how frequently specific types of waste are being introduced and how much room there is for improvement, thereby generating suggestions for product improvements.

[0634] Step 6:

[0635] Users earn eco-points by properly classifying their waste. These points are managed through a smartphone application, and accumulated points can be used for discount coupons or product purchases, contributing to increased environmental awareness among users.

[0636] Step 7:

[0637] The server automatically generates improvement suggestions based on the analysis results and notifies designated manufacturers and stakeholders. These suggestions are sent via email or a web portal and used as feedback for product improvement.

[0638] (Example 1)

[0639] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0640] In modern society, vast amounts of waste are generated, and its proper management has a significant impact on the global environment. However, effective waste classification and improved recycling rates require active participation from users, as well as product improvement suggestions from manufacturers. Given this situation, there is a lack of systems to promote efficient waste management and recycling activities.

[0641] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0642] In this invention, the server includes a device having the function of acquiring image information and identifying organic matter based on the image information, a device having the function of classifying substances based on the identification results of the organic matter, and a device having the function of reading identification tags attached to substances and acquiring usage history information of the substances. This enables users to accurately classify waste without being aware of it, generates appropriate improvement suggestions for producers, and promotes the efficiency of waste management and recycling activities.

[0643] "Image information" refers to digital data that visually captures the appearance of a substance and is used for identification and classification.

[0644] "Organic matter" refers to substances that include materials derived from living organisms and their processed products, and are classified as recyclable resources.

[0645] An "identification tag" is a small identification device attached to an object that has the function of electronically storing information such as usage history.

[0646] "Usage history information" refers to a series of pieces of information about a substance from its manufacture to its use and disposal, and is data used to understand the product's lifecycle.

[0647] A "device" refers to a combination of hardware and software designed to perform a specific function.

[0648] A "producer" refers to a corporation or individual responsible for manufacturing or supplying a product, and is the party to whom suggestions regarding product improvements should be received.

[0649] This invention is a system for achieving efficient waste management through the cooperation of a terminal, a server, and a user. Specifically, the terminal is installed in a smart trash can and is equipped with a sensor that activates when waste is placed inside and a camera for capturing images. This camera acquires image information of the waste and transmits that data to the server in real time. Through this operation, the waste is digitized without the user having to be particularly aware of it.

[0650] The server functions as a device that executes machine learning algorithms using the received image information. For example, it applies known deep learning models such as ResNet and VGG to identify the type of waste and classify it into categories such as "plastic," "paper," and "metal" by comparing it with a database. Furthermore, the terminal uses an RFID reader to read the identification tags attached to the materials and obtain information on the material's usage history. This provides information for a detailed analysis of the product's lifecycle.

[0651] The collected identification and usage history information is integrated on a server, and improvement suggestions are generated for specific producers. These suggestions include specific improvement plans aimed at increasing the efficiency of waste recycling and are communicated to producers or relevant parties through a predetermined communication method. This facilitates improvements during the product development stage.

[0652] Users are rewarded for correctly classifying their waste, and these rewards can be managed through a dedicated application or web portal. Users can use these rewards to exchange for benefits within their local community.

[0653] As a concrete example, when a user places a plastic bottle into a smart trash can, the device takes a picture of it with its camera, and the image sent to the server is analyzed and classified as recyclable "plastic." The RFID tag attached to the plastic bottle is read, and details such as "Manufacturer: General Incorporated Association, Manufacturing Date: Specified Date" are obtained, and the product's recycling data is updated based on this information. An example of a prompt to be input into the generating AI model would be, "Explain how paper waste is identified in the automated waste classification system. Include the name of the relevant machine learning algorithm and processing procedure."

[0654] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0655] Step 1:

[0656] The device uses sensors to detect when waste is placed in the smart trash can. This detection triggers a camera to activate, taking an image of the waste and generating image data. The input is the physical placement of waste, and the output is digital image data. The generated image data is immediately sent to the server.

[0657] Step 2:

[0658] The server receives image data from the terminal. Next, it inputs the image data into a machine learning algorithm to identify the type of waste. This process utilizes generative AI models such as ResNet and VGG to analyze image features and classify the waste into categories such as "plastic," "paper," and "metal." The input is image data, and the output is the category information of the classified waste.

[0659] Step 3:

[0660] The terminal reads the identification tag attached to the waste using an RFID reader. This retrieves product lifecycle data, such as manufacturer and manufacturing date. The input is the RFID tag attached to the waste, and the output is the usage history information contained within the tag. This information is sent to a server for product tracking and analysis.

[0661] Step 4:

[0662] The server integrates the category information of classified waste with usage history information obtained from RFID readers. Based on this information, it performs analysis to identify the recycling rate of products and areas for improvement at a specific producer. Based on the analysis results, it automatically generates improvement suggestions. The input is integrated information, and the output is improvement suggestions for producers.

[0663] Step 5:

[0664] When a user correctly sorts waste, the server calculates and awards eco-points as a corresponding reward. These points are managed through a dedicated application or web portal, and users can use them to exchange for local benefits. Inputs are the sorting results and user ID, and output is the awarded eco-point information.

[0665] Through these steps, the system enables efficient waste management and supports active user participation and product improvement.

[0666] (Application Example 1)

[0667] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0668] Proper waste classification is crucial for urban environments and business operations, but the cumbersome process of individual users consciously classifying waste leads to misdisposal and low recycling rates. Furthermore, insufficient user motivation to promote recycling activities necessitates more efficient waste management. Another challenge is the lack of timely feedback to manufacturers regarding product improvements.

[0669] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0670] In this invention, the server includes an acquisition means for acquiring image information and identifying waste, a classification means for classifying waste based on the identification results, a reading means for acquiring and analyzing lifecycle information, and a presentation means for displaying waste classification information and eco-points on the user's device. This enables real-time waste classification, effective utilization of an eco-point system, and product improvement through rapid feedback.

[0671] "Acquisition means" refers to a device that has the function of acquiring image information related to waste and performing identification based on that data.

[0672] A "classification device" is a device that analyzes acquired image information of waste and classifies it into the appropriate category.

[0673] A "reading device" is a device that has the function of reading identification tags attached to waste and obtaining its lifecycle information.

[0674] The "analysis means" is a device that performs an analysis by comprehensively analyzing the acquired lifecycle information and identification information, and has the function of generating product improvement suggestions for manufacturers.

[0675] A "presentation means" is a device that has the function of visually displaying waste classification information and eco-point status in real time on the user's device.

[0676] "Eco-points" are points awarded to users when they properly classify and dispose of waste, and the system is designed to incentivize users' environmental protection activities.

[0677] The system implementing this invention consists of three components: a server, a terminal, and a user. For the entire system to function smoothly, each of these components must fulfill its respective role. First, the terminal uses sensors and cameras installed in the smart trash can to acquire image data when waste is placed inside. Specifically, the camera takes high-resolution images of the waste, and the RFID reader reads the identification tag attached to the waste. This information is immediately transmitted to the server.

[0678] The server uses image recognition libraries such as OpenCV and TensorFlow to process the received image data. This classifies the waste into categories such as "plastic," "paper," and "metal." Next, the server stores the read lifecycle information in a database and performs analysis. The analysis results are used to evaluate the traceability and recyclability of the product. Furthermore, based on the analysis data, the server awards eco-points to users who correctly classify their waste.

[0679] Users can check eco-points and waste classification information in real time using their smartphones or smart glasses. The system visually displays the classification results and suggestions for using eco-points on these devices. For example, if a user correctly recycles plastic, a message such as "Your plastic has been correctly recycled. Eco-points have been awarded." will be displayed.

[0680] As a concrete example, when a user disposes of waste at home, using smart glasses would provide real-time information such as, "This is metal. Recycling it will increase your eco-points." Such a system would create an incentive for users to participate more actively in recycling activities.

[0681] Furthermore, an example of a prompt message from the AI ​​model in this system is a text-based message such as, "Based on the photo and RFID information of this waste, explain how to classify it and calculate and present the appropriate eco-points." This incentivizes the server to perform appropriate classification and rewarding.

[0682] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0683] Step 1:

[0684] The terminal uses a camera mounted on the smart trash can to acquire image data when waste is placed inside. This acquired image becomes the input. The input image data is immediately transmitted from the terminal to the server for identification. As preparation for transmission, a specific operation is performed to convert the image data format.

[0685] Step 2:

[0686] The server identifies the type of waste from the received image data. This process uses image recognition libraries such as OpenCV or TensorFlow, with the image data as input. The server performs pattern analysis on the data and obtains output classified into categories such as "plastic," "metal," and "paper."

[0687] Step 3:

[0688] The terminal uses an RFID reader to read the identification tag attached to the waste. The reading result is sent to the server as input. Specifically, this involves decoding the tag information and saving it to a database.

[0689] Step 4:

[0690] The server performs analysis based on the lifecycle information and identification results. It receives lifecycle information and identification results as input, performs data analysis, and outputs an optimized recycling method. These results are used for future improvements and generate feedback reports for manufacturers.

[0691] Step 5:

[0692] The server calculates and awards eco-points for waste correctly classified by the user. The server uses the classification results as input, performs calculations to award eco-points, and outputs them as the user's score. Specifically, point records and account integration are performed.

[0693] Step 6:

[0694] Users can check waste classification information and eco-point status through an application on their device. Based on output data transmitted from the server, the device visualizes and presents the information to the user. Specifically, it performs real-time information updates and notifications.

[0695] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0696] This invention is a system that combines acquisition means, classification means, reading means, analysis means, and emotion recognition engine to improve the efficiency of waste management. This system assists the user in properly classifying waste while simultaneously analyzing the user's emotional state and providing feedback and adjusting the process.

[0697] First, the device incorporates sensors into the smart trash can to detect when waste is placed inside. At this time, a camera is used to acquire image data. This system automatically collects data every time the user disposes of waste. This image data is transmitted to a server in real time, ready for analysis.

[0698] The server processes the received image data using an AI image recognition module to identify the type of waste. The identified data is used not only for waste classification but also for overall analysis. The terminal also uses an RFID reader to obtain lifecycle data from identification tags attached to the waste and transmits this data to the server.

[0699] Furthermore, the server uses an emotion recognition engine to analyze the user's facial expressions and voice tone captured by the camera to identify the user's emotional state. Based on this emotional data, the system optimizes the interface and feedback provided to the user. For example, if it determines that the user is experiencing stress during waste sorting, it provides guidance and assistance to make the process more intuitive.

[0700] Furthermore, users who properly sort their waste will be awarded eco-points. These points serve as a means to raise users' environmental awareness and promote recycling activities. Eco-points are managed through a dedicated application and can be used for various purposes.

[0701] Ultimately, the server generates suggestions for product improvement and environmental impact reduction based on all identification and analysis data, and notifies the relevant manufacturers and responsible parties. This notification process is carried out quickly and efficiently via email and a web portal.

[0702] Thus, by implementing the present invention, it is expected that user convenience in waste management will be enhanced and that the realization of a sustainable society will be contributed to.

[0703] The following describes the processing flow.

[0704] Step 1:

[0705] The terminal uses sensors to detect when waste is dumped. When waste is discarded, the camera automatically activates and takes an image. This image records the appearance of the discarded waste. The captured image data is sent to a server for analysis.

[0706] Step 2:

[0707] The server passes the received image data to an AI image recognition module, which analyzes the type of waste. For example, it is classified into categories such as "paper," "plastic," and "metal." This classification information is used to support the proper disposal of the waste.

[0708] Step 3:

[0709] The terminal uses an RFID reader to read identification tags attached to waste. Manufacturer and product information is obtained from these tags and sent to a server as lifecycle data. This makes it possible to track the origin and characteristics of the waste.

[0710] Step 4:

[0711] The server analyzes the identification data and lifecycle information of the collected waste. Specifically, it performs trend analysis and automatically generates improvement suggestions for manufacturers. This provides concrete measures for manufacturers to reduce their environmental impact.

[0712] Step 5:

[0713] The server uses an emotion recognition engine to analyze the user's facial expressions and voice patterns to assess their emotional state. For example, if the user is stressed, the server adjusts the interface to simplify operations and improve the user experience.

[0714] Step 6:

[0715] Users are awarded eco-points for properly classifying their waste. These points are managed through a dedicated application, allowing for easy checking and use. Eco-points can be used for discounts on reusable products and other benefits, increasing users' contribution to the environment.

[0716] Step 7:

[0717] Based on the analysis and sentiment recognition results, the server notifies the relevant manufacturer or person in charge of product improvement suggestions. Notifications are made via email or a dedicated portal, creating a rapid feedback loop. This facilitates continuous product improvement.

[0718] (Example 2)

[0719] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0720] In waste management, it is necessary to efficiently identify and classify waste appropriately, provide feedback that takes user sentiment into consideration, and promote recycling activities. Furthermore, it is crucial to use the acquired data to drive product improvements in the manufacturing sector. Current systems suffer from insufficient waste identification and classification, as well as user support, leading to delays in suggestions and notifications to the manufacturing sector.

[0721] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0722] In this invention, the server includes an acquisition means having the function of acquiring image data and identifying waste based on the image data; a reading means having the function of reading information tags associated with the waste and acquiring waste usage history information; and an analysis means having the function of identifying the user's emotional state based on camera images and audio data and optimizing the interface and feedback content based on the emotional state. This enables appropriate identification and classification of waste, provides appropriate feedback according to the user's emotional state, and facilitates rapid improvement suggestions in the manufacturing field.

[0723] "Image data" refers to visual information acquired to represent the appearance of waste.

[0724] "Acquisition means" refers to a device or function for collecting image data.

[0725] "Classification means" refers to a device or function for dividing identified waste into specific categories.

[0726] "Reading means" refers to a device or function for detecting information tags associated with waste and collecting that information.

[0727] An "information tag" is an identifier attached to waste materials to record information about their usage history.

[0728] "Usage history information" refers to information about the waste's manufacturer, usage route, and recovery status.

[0729] "Analysis means" refers to a device or function for analyzing the condition of waste and users based on acquired data.

[0730] "Emotional state" refers to information that represents the user's emotions and is identified through facial recognition and voice analysis.

[0731] An "interface" is a mechanism or method for a user to interact with a system.

[0732] "Feedback" refers to information and instructions provided to a user, used to support their actions.

[0733] An "improvement suggestion" is advice or recommendation based on data analysis to encourage improvements to products and processes in the manufacturing sector.

[0734] "Network infrastructure" refers to the communication infrastructure used to transmit data.

[0735] This invention is a system for improving the efficiency of waste management, and includes acquisition means, classification means, reading means, analysis means, and feedback optimization function. When a user disposes of waste, this system acquires image data and audio data, and identifies and analyzes the type of waste based on that data. Furthermore, it analyzes the user's emotional state and provides feedback tailored to the user.

[0736] Specifically, the device incorporates sensors into the smart trash can to detect when waste is placed inside. These sensors include infrared and pressure sensors. Once waste is detected, a camera activates to capture high-resolution images of the waste. This image data is transmitted to a server in real time. The server uses a generative AI model to analyze the image data and identify the type of waste. For example, it may be classified as plastic, metal, or paper. This classification result is stored in a database and used to improve waste management.

[0737] Simultaneously, the terminal reads the information tags attached to the waste and obtains lifecycle data. This data, which includes the product's manufacturer and usage history, is transferred to a server and used for further analysis.

[0738] Furthermore, the server incorporates an emotion recognition engine that uses the camera and voice input capabilities to analyze the user's facial expressions and tone of voice. The emotion data obtained from this analysis is used to optimize the user interface. If the user shows difficulty with waste classification, the system provides assistance and offers instructions to improve the classification process.

[0739] Furthermore, an eco-point system has been introduced, where users are awarded points for correctly classifying their waste. These points not only raise users' environmental awareness but can also be exchanged for rewards. Points are managed using a dedicated application and can be exchanged for things like discount coupons at supermarkets.

[0740] Ultimately, the server generates improvement suggestions based on all identification and sentiment data and notifies managers in the relevant manufacturing sectors. This notification is made via email or a web portal to enable rapid feedback. For example, a prompt for the generating AI model might say, "Suggest ways to optimize the recycling of plastic bottles using a waste management system."

[0741] Thus, this invention improves the efficiency of waste management while simultaneously personalizing the user experience and contributing to the realization of a sustainable society.

[0742] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0743] Step 1:

[0744] The device detects waste being placed in the smart trash can using a sensor built into the trash can. This sensor uses infrared technology and emits a signal when waste passes through the opening. The input is a physical change recognized by the sensor. By detecting this change, an output is obtained that recognizes that waste has been placed in the trash can.

[0745] Step 2:

[0746] The device activates its camera and acquires image data of the waste. Multiple images are taken from different angles and stored as high-resolution visual information. The input is the physical shape and color of the waste, and the output is image data. This data is transmitted to the server in real time.

[0747] Step 3:

[0748] The server analyzes the received image data using a generation AI model. The input is the transmitted image data, and the AI ​​identifies the type of waste based on a deep learning algorithm. This analysis generates waste category information as output. For example, it might be classified as plastic, metal, paper, etc.

[0749] Step 4:

[0750] The terminal reads information tags attached to waste and acquires lifecycle data using radio frequency identification (RFID) technology. The input is the RFID tag, and the output is sent to the server as usage history information. This information provides detailed data regarding the origin and usage route of the waste.

[0751] Step 5:

[0752] The server uses an emotion recognition engine to analyze the user's facial expressions and voice tone captured by the camera. Input includes the user's facial image and voice data. The output identifies the user's emotional state, and the system uses this data to provide corresponding feedback.

[0753] Step 6:

[0754] Based on the user's emotional state, the system optimizes feedback and the interface. Specifically, if it determines that the user is experiencing stress, it displays guidelines to simplify operation or provides voice assistance. The input is the user's emotional data, and the output is optimized user interface information.

[0755] Step 7:

[0756] The system awards eco-points to users who correctly classify their waste. Input includes information about the accuracy of the classification, and the points are recorded in the user's account as output. These points can be managed and viewed through a dedicated application.

[0757] Step 8:

[0758] The server analyzes all collected data and generates improvement suggestions for the manufacturing sector. Inputs include identification results and usage history information, while output lists specific actions as improvement suggestions. These suggestions are then communicated to relevant parties via email and a web portal.

[0759] (Application Example 2)

[0760] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0761] In today's mass-consumption society, proper waste classification and recycling are crucial issues from an environmental protection perspective. However, users often struggle to accurately classify waste, and the process can be stressful. Therefore, there is a need for support systems to improve the efficiency of waste management.

[0762] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0763] In this invention, the server includes an acquisition means having the function of acquiring image information and identifying waste based on said image information; a classification means having the function of classifying the waste based on the identification result of said waste; and an analysis means having the function of analyzing the user's emotional state and optimizing the user interface based on that data. This makes it easier for the user to intuitively classify waste and to provide appropriate feedback to further increase motivation.

[0764] "Image information" refers to visual data acquired through cameras and other imaging devices.

[0765] "Waste" refers to used products and unwanted materials that require proper disposal or recycling.

[0766] "Acquisition means" refers to a function for collecting image information of waste and using it within the system.

[0767] A "classification method" is a function that identifies the type of waste based on the image information acquired and sorts it into the appropriate category.

[0768] The "reading means" refers to a function that scans identification tags attached to waste materials to obtain their lifecycle information.

[0769] "Analysis means" refers to a function that processes data based on lifecycle information and identification results, and generates suggestions for improving the user interface.

[0770] "Emotional state" refers to the psychological state analyzed from the user's facial expressions, voice, etc., and is an element that the system uses to optimize interaction.

[0771] An "interface" is a user interaction function that allows users to interact with a system and exchange information.

[0772] "Eco-points" are rewards given to users who properly classify their waste, and serve as an incentive to promote environmental awareness.

[0773] In the system implementing this invention, a camera installed on a terminal first collects image information of the waste. The acquired image information is sent to a server, where an AI image recognition model is used to identify the type of waste. Specific software used includes PyTorch and TensorFlow. Based on this identification data, the server appropriately classifies the waste.

[0774] The terminal also features an RFID reader to read identification tags attached to waste items, acquiring lifecycle information. The acquired lifecycle information and identification data are analyzed on a server. During this process, since the user's emotional state affects the user interface, an emotion recognition engine is used to analyze the user's facial expressions and voice to identify their state. Based on the analysis results, support information and feedback are provided to help the user process waste more easily.

[0775] Furthermore, users who correctly classify their waste are awarded eco-points, which are managed through a dedicated application. This point system serves as an incentive to raise users' environmental awareness. The server also has the ability to generate suggestions for product improvement and environmental impact reduction based on all identification and analysis data, and notify relevant organizations. These notifications are sent via email and a web portal.

[0776] A concrete example is a system installed at a recycling center in a certain city. When a user disposes of a plastic bottle and records the process through smart glasses, the bottle is immediately identified and the interface is adjusted accordingly. If the user shows signs of confusion based on sentiment analysis, the system provides simple guidance. Furthermore, if the disposal is done properly, eco-points are added to the user's smartphone app, encouraging community use.

[0777] Examples of prompts for a generative AI model:

[0778] "Please explain, with examples, how a smart recycling assistant app can improve residents' waste disposal habits."

[0779] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0780] Step 1:

[0781] The device uses a sensor to detect when waste is placed in a trash can and a camera to capture an image of the waste. This image is immediately sent to a server and processed as image information. The input is image data from the camera, and the output is digital data containing that information.

[0782] Step 2:

[0783] The server analyzes the received image information using an AI image recognition module to identify the type of waste. This process inputs the acquired image data into an AI model and outputs an identification label as a classification result. Specifically, an identification algorithm using TensorFlow is in operation.

[0784] Step 3:

[0785] Upon receiving the identification result, the terminal uses an RFID reader to read the identification tag attached to the waste. It then obtains the waste's lifecycle information from this tag and transmits it to the server. The input is the signal from the identification tag, and the output is the lifecycle information derived from that data.

[0786] Step 4:

[0787] The server activates an emotion recognition engine based on lifecycle information and identification results, and analyzes the user's emotional state using camera and audio data. The input is the user's facial expressions and audio data, and the output is an evaluation of their emotional state. OpenCV and the emotion_recognition library are used in this process.

[0788] Step 5:

[0789] Based on the analyzed data, the server optimizes the user interface and provides support information in real time. If the user is experiencing stress, it presents more user-friendly guides and assistance. The input is emotional evaluation, and the output is an optimized interface and feedback.

[0790] Step 6:

[0791] If a user correctly classifies their waste, the server calculates eco-points and adds them to the user's account. This process calculates the number of points based on the classification results and sends that information to the user's application. The input is the accuracy of the classification, and the output is a notification of point addition.

[0792] Step 7:

[0793] The server integrates all the data and generates suggestions for product improvements and reducing environmental impact. These suggestions are communicated to relevant organizations via email and a web portal. The input is the entire analyzed data, and the output is the suggestion notification message.

[0794] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0795] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0796] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0797] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0798] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0799] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0800] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0801] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0802] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0803] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0804] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0805] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0806] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0807] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0808] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0809] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0810] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0811] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0812] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0813] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0814] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0815] The following is further disclosed regarding the embodiments described above.

[0816] (Claim 1)

[0817] An acquisition means having the function of acquiring image information and identifying waste based on said image information,

[0818] A classification means having the function of classifying waste based on the identification result of the waste,

[0819] A reading means having the function of reading identification tags attached to waste and obtaining lifecycle information of the waste,

[0820] An analysis means having the function of analyzing data based on the lifecycle information and identification results and generating improvement suggestions for a specific manufacturer,

[0821] A system that includes this.

[0822] (Claim 2)

[0823] The system according to claim 1, which has a function to award points to users who correctly classify waste based on acquired data.

[0824] (Claim 3)

[0825] The system according to claim 1, further comprising a function for transmitting generated improvement suggestions to the relevant manufacturer or person in charge using a predetermined means of communication.

[0826] "Example 1"

[0827] (Claim 1)

[0828] A device having the function of acquiring image information and identifying organic matter based on said image information,

[0829] A device having the function of classifying substances based on the identification result of the organic substance,

[0830] A device having the function of reading identification tags attached to a substance and acquiring information on the substance's usage history,

[0831] A device having the function of analyzing information based on the usage history information and identification results and generating improvement suggestions for a specific producer,

[0832] A device that has a function to reward users when they correctly classify organic matter based on the results of the analysis,

[0833] A system that includes this.

[0834] (Claim 2)

[0835] The system according to claim 1, having a function to transmit the generated improvement proposals, based on the acquired information, to relevant producers or stakeholders using a predetermined communication method.

[0836] (Claim 3)

[0837] The system according to claim 1, which stores acquired reward information and provides functions for users to manage and utilize rewards.

[0838] "Application Example 1"

[0839] (Claim 1)

[0840] An acquisition means having the function of acquiring image information and identifying waste based on said image information,

[0841] A classification means having the function of classifying waste based on the identification result of the waste,

[0842] A reading means having the function of reading identification tags attached to waste and obtaining lifecycle information of the waste,

[0843] An analysis means having the function of analyzing data based on the lifecycle information and identification results and generating improvement suggestions for a specific manufacturer,

[0844] A display means that has the function of displaying waste classification information on the user's device and displaying the status of eco-points in real time,

[0845] A system that includes this.

[0846] (Claim 2)

[0847] The system according to claim 1, which has a function to award points to users who correctly classify waste based on acquired data, and to present point usage suggestions on the user's device in real time.

[0848] (Claim 3)

[0849] The system according to claim 1, further comprising the function of transmitting the generated improvement suggestions to the relevant manufacturer or person in charge using a predetermined means of communication, and presenting the improvement information to the user's device.

[0850] "Example 2 of combining an emotion engine"

[0851] (Claim 1)

[0852] An acquisition means having the function of acquiring image data and identifying waste based on said image data,

[0853] A classification means having the function of classifying waste based on the identification result of the waste,

[0854] A reading means having the function of reading information tags associated with waste and obtaining waste usage history information,

[0855] An analysis means having the function of identifying the user's emotional state based on camera images and audio data, and optimizing the interface and feedback content based on said emotional state,

[0856] An analysis means having the function of analyzing data based on usage history information, identification results, and emotional state, and generating improvement suggestions for a specific manufacturing field,

[0857] A system that includes this.

[0858] (Claim 2)

[0859] The system according to claim 1, which has a function to reward users who have correctly classified waste based on acquired data and user operation results.

[0860] (Claim 3)

[0861] The system according to claim 1, having a function to transmit generated improvement suggestions to the relevant manufacturing field or manager using a predetermined network means.

[0862] "Application example 2 when combining with an emotional engine"

[0863] (Claim 1)

[0864] An acquisition means having the function of acquiring image information and identifying waste based on said image information,

[0865] A classification means having the function of classifying waste based on the identification result of the waste,

[0866] A reading means having the function of reading identification tags attached to waste and obtaining lifecycle information of the waste,

[0867] An analysis means having the function of analyzing the emotional state of the user and optimizing the user interface based on that data,

[0868] A system that includes this.

[0869] (Claim 2)

[0870] The system according to claim 1, which has a function to award points to users who correctly classify waste based on acquired data.

[0871] (Claim 3)

[0872] The system according to claim 1, further comprising a function for transmitting generated improvement suggestions to the relevant manufacturer or person in charge using a predetermined means of communication. [Explanation of Symbols]

[0873] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. An acquisition means having the function of acquiring image information and identifying waste based on said image information, A classification means having the function of classifying waste based on the identification result of the waste, A reading means having the function of reading identification tags attached to waste and obtaining lifecycle information of the waste, An analysis means having the function of analyzing data based on the lifecycle information and identification results and generating improvement suggestions for a specific manufacturer, A display means that has the function of displaying waste classification information on the user's device and displaying the status of eco-points in real time, A system that includes this.

2. The system according to claim 1, which has a function to award points to users who have correctly classified waste based on acquired data, and to present point usage suggestions on the user's device in real time.

3. The system according to claim 1, further comprising the function of transmitting the generated improvement suggestions to the relevant manufacturer or person in charge using a predetermined communication means, and presenting the improvement information to the user's device.

Images