System

Abstract

A system is provided.SOLUTION: A system comprising: image recognition means; generative artificial intelligence means; robotics means; communication means; and interface means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Proper waste sorting at home and incineration plants is not easy and requires a lot of time and effort. The increase in mixed waste, the difficulty of accurate sorting, and the complexity of sorting methods are particularly problematic. Furthermore, traditional manual sorting methods inevitably lead to a decline in recycling efficiency due to labor shortages and missorting. Given this background, there is a need for automated systems using image recognition technology and generative artificial intelligence to streamline waste sorting and improve recycling rates. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a waste sorting system that includes an image recognition means, a generative artificial intelligence means, a robotics means, a communication means, and an interface means.

[0006] Specifically, an image of the waste is acquired using image recognition means, and the type of waste is identified using generative artificial intelligence means. The identified waste information is transported to the appropriate sorting zone by robotics means. Furthermore, accurate sorting methods are provided to users by utilizing waste sorting devices for home and waste disposal plants, smartphone applications, QR codes or RFID tags, and audio guidance means.

[0007] This will enable accurate and efficient waste sorting, which will help improve recycling rates and promote the realization of a sustainable society.

[0008] "Image recognition means" refers to technology and equipment that uses cameras and sensors to capture images of garbage, analyzes the data, and identifies the type of garbage.

[0009] "Generative AI means" refers to an AI technology that uses acquired image data and other input data to identify the type of garbage and improves accuracy through self-learning.

[0010] "Robotics means" refers to mechanical devices such as robotic arms or conveyor belts that transport identified waste to the appropriate sorting zone.

[0011] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[0012] An "interface means" is a device or interface, such as an operation panel or software application, that facilitates user interaction with the system.

[0013] A "household garbage sorting device" is a small-sized garbage sorting device that assists in garbage sorting at home.

[0014] A "waste sorting device for an incineration plant" is a large-scale waste sorting device installed in an incineration plant.

[0015] The "smartphone application" is a mobile application that allows users to take a photo of garbage and receive suggestions on how to identify the type of garbage and sort it using artificial intelligence.

[0016] "QR code" is a type of two-dimensional barcode, a technology that can store a lot of information in a relatively small code.

[0017] An "RFID tag" is a tag that uses radio frequency identification technology and can identify and track items.

[0018] "Voice guidance means" refers to a technique or device for providing sorting instructions or other guidance to users by voice within the system.

[0019] A "garbage truck" is a vehicle that collects garbage from homes and businesses and transports it to a disposal site or recycling facility.

[0020] "Self-learning function" refers to the ability of the generative AI to learn and improve its accuracy based on past data and user feedback.

[0021] "Feedback" refers to the reaction or information provided by the user to the system's classification results, etc. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0023] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0026] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0027] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0028] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0030] [First embodiment]

[0031] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0032] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0035] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0038] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0042] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0043] 1. System Overview

[0044] The present invention is a system for automatically separating waste in homes and waste disposal plants, thereby improving recycling rates. The system includes image recognition means, generative artificial intelligence means, robotics means, communication means, and interface means, which work together to make waste separation more efficient.

[0045] 2. System Configuration

[0046] Image recognition method: Images of garbage are captured using cameras and sensors, and the data is analyzed to identify the type of garbage.

[0047] Generative AI means: Identifies the type of garbage using acquired image data, and improves accuracy through self-learning functions.

[0048] Robotic means: Devices such as robotic arms and conveyor belts that transport identified waste to the appropriate sorting zone.

[0049] Communications: The network and communications protocols used to send and receive data to and from servers and other devices.

[0050] Interface means: An operating panel or software application that facilitates user interaction with the system.

[0051] 3. Program Processing and Operation

[0052] Garbage image acquisition and recognition

[0053] 1. The device (e.g., a camera mounted on a smart trash can or a garbage truck) captures an image every time garbage is dropped in and sends the image data to a server.

[0054] 2. The server inputs the received image data into a generative AI model to identify the type of garbage.

[0055] 3. The server sends the identification result to the robotic means.

[0056] Sorting garbage

[0057] 4. Based on the identification results from the server, the terminal (robotics system) operates the robotic arm and conveyor belt to transport the waste to the appropriate sorting zone.

[0058] Home and waste incineration plant systems

[0059] 5. The terminal will provide voice guidance and display the type of garbage, and guide the user on the appropriate sorting method.

[0060] 6. Users can also use a smartphone app to take photos of trash and send them to the server, which then notifies the user of the image identification results and sorting instructions.

[0061] 4. Specific Examples

[0062] Example 1: Smart trash can for home use

[0063] The device (a camera built into the smart trash can) captures an image of the plastic bottle.

[0064] The terminal transmits the image data to the server.

[0065] The server uses generative AI to identify it as a plastic bottle and sends instructions to the smart trash can to sort it into a plastic sorting zone.

[0066] The terminal opens the plastic lid based on the instructions and guides the user.

[0067] Example 2: Robotics system for waste incineration plants

[0068] The terminal (a camera on the conveyor belt at the waste disposal plant) captures images of the mixed waste.

[0069] The terminal transmits the image data to the server.

[0070] The server uses a generative AI to identify each type of garbage.

[0071] The server sends each identification result to the robotics system.

[0072] A robotic means (robot arm) transports each waste to the appropriate sorting zone according to the identification results.

[0073] 5. Feedback and self-study

[0074] Users can send feedback through the smart trash can or smartphone app, and if the sorting is incorrect, the user's feedback will be reflected in the system.

[0075] The server collects feedback data and updates the generative AI model, which continuously improves its classification accuracy.

[0076] In this way, the system of the present invention improves the efficiency of waste sorting at homes and waste disposal plants, and realizes highly accurate recycling.

[0077] The processing flow will be explained below.

[0078] Processing flow for smart household trash cans

[0079] Step 1: Image capture of the garbage

[0080] Operation details:

[0081] The device (a camera built into the smart trash can) takes an image every time trash is dropped in.

[0082] Step 2: Sending images

[0083] Operation details:

[0084] The terminal transmits the captured image data to the server.

[0085] Step 3: Image Identification (using generative AI)

[0086] Operation details:

[0087] The server inputs the received image data into a generative AI model to identify the type of garbage.

[0088] Step 4: Notification of identification results

[0089] Operation details:

[0090] The server transmits the identification result to the terminal.

[0091] Step 5: Providing guidance on waste sorting

[0092] Operation details:

[0093] The terminal opens the lid of the appropriate sorting zone based on the type of waste identified.

[0094] The terminal guides the user on how to separate the waste by voice guidance or display on the screen.

[0095] Processing flow in a robotics system for waste incineration plants

[0096] Step 1: Image capture of the garbage

[0097] Operation details:

[0098] The terminal (a camera on the conveyor belt at the waste disposal plant) takes images of the waste as it flows by.

[0099] Step 2: Sending images

[0100] Operation details:

[0101] The terminal transmits the captured image data to the server.

[0102] Step 3: Image Identification (using generative AI)

[0103] Operation details:

[0104] The server inputs the received image data into a generative AI model to identify each type of garbage.

[0105] Step 4: Notification of identification results

[0106] Operation details:

[0107] The server sends each identification result to the robotics system.

[0108] Step 5: Sorting your trash

[0109] Operation details:

[0110] The terminal (robot arm) transports each piece of garbage to the appropriate sorting zone based on the identification results from the server.

[0111] Processing flow for smartphone app integration

[0112] Step 1: Photograph the litter

[0113] Operation details:

[0114] Users take photos of the trash using a smartphone app.

[0115] Step 2: Send your photos

[0116] Operation details:

[0117] The terminal transmits the photographed photo data to the server.

[0118] Step 3: Image Identification (using generative AI)

[0119] Operation details:

[0120] The server inputs the received image data into a generative AI model to identify the type of garbage.

[0121] Step 4: Propose a sorting method

[0122] Operation details:

[0123] Based on the identification results, the server sends the appropriate sorting method to the terminal (smartphone app).

[0124] The terminal displays the sorting method to the user.

[0125] Feedback and self-learning process flow

[0126] Step 1: Submit your feedback

[0127] Operation details:

[0128] Users can provide feedback on their sorting results via a smartphone app or smart trash can.

[0129] Step 2: Receive and analyze feedback

[0130] Operation details:

[0131] The server analyzes the received feedback data.

[0132] Step 3: Update the AI ​​model

[0133] Operation details:

[0134] The server updates the generative AI model based on the feedback data.

[0135] Step 4: Apply the model

[0136] Operation details:

[0137] The server will use the updated AI model and apply it to future image recognition processes.

[0138] Through these processing steps, the sorting accuracy of the entire system will improve, aiming to increase recycling rates and realize a sustainable society.

[0139] Example 1

[0140] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0141] Conventional waste sorting at homes and waste disposal plants relies on manual labor or simple machines, resulting in low sorting accuracy and limiting the improvement of recycling rates. Manual sorting is also labor-intensive and time-consuming, and has many issues with efficiency. Furthermore, when incorrect sorting occurs, it is difficult to incorporate user feedback into the system, and the system lacks a self-learning function. There is a need for a system that can solve these problems and achieve high-precision, efficient waste sorting.

[0142] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0143] In this invention, the server includes a means for transmitting image data from the terminal to the server, a means for analyzing image data in the server, and a means for transmitting identification results from the server to the robotics means. This enables highly accurate identification of waste types. The server also includes a means for receiving feedback data from users and a means for self-learning and updating the generative artificial intelligence model, allowing for continuous improvement in identification accuracy through feedback. Furthermore, the robotics means automatically transports waste to the sorting zone, significantly improving work efficiency. This system can be used for both home and waste disposal plants, contributing to increased recycling rates.

[0144] "Image recognition means" refers to a device or software that analyzes acquired image data and identifies the type of garbage.

[0145] "Generative artificial intelligence means" refers to an AI model and related system that uses acquired image data to identify the type of garbage and improves identification accuracy through self-learning functions.

[0146] "Robotics means" refers to devices such as robotic arms and conveyor belts that transport identified waste to the appropriate sorting zone.

[0147] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[0148] "Interface means" refers to an operation panel or software application that facilitates user interaction with the system.

[0149] The "means for transmitting image data from the terminal to the server" is a function for transmitting image data captured by the terminal to the server in real time.

[0150] "Image data analysis means on the server" is a function that inputs the image data received by the server into a generative AI model, analyzes it, and identifies the type of garbage.

[0151] The "means for transmitting identification results from the server to the robotics means" is a function for transmitting the identification results analyzed by the server to the robotics means.

[0152] The "means for receiving feedback data from users" is a function that receives feedback data provided by users via smartphone apps or the like on the server.

[0153] The "self-learning update means for the generative AI model" is a function that updates the learning of the generative AI model using received feedback data to improve the recognition accuracy.

[0154] System Overview

[0155] The present invention is a system for automatically separating waste at homes and waste disposal plants to improve recycling rates. This system operates by linking together image recognition means, generative artificial intelligence means, robotics means, communication means, and interface means.

[0156] Hardware and software used

[0157] Image recognition method: Images of garbage are captured using cameras or sensors, and the data is analyzed. Specifically, a high-resolution camera (e.g., Sony's IMX577 sensor) is used.

[0158] Generative AI method: The acquired image data is input into a generative AI model (e.g., a deep learning model using TensorFlow or PyTorch) to identify the type of garbage.

[0159] Robotic means: using robotic arms or conveyor belts that transport identified waste to the appropriate sorting zone. Examples include robotic arms from KUKA and Fanuc.

[0160] Communication method: Uses a network (e.g., Wi-Fi module and communication protocols such as HTTP, WebSocket, and MQTT) to send and receive data to and from servers and other devices.

[0161] Interface means: An operation panel or software application that makes it easy for the user to interact with the system. Examples include Android and iOS applications.

[0162] Image acquisition and identification of debris

[0163] 1. The device uses cameras installed in smart trash bins or on conveyor belts at waste disposal plants to capture images every time trash is dropped in.

[0164] 2. The device transmits the acquired image data to the server in real time.

[0165] 3. The server inputs the received image data into a generative AI model to identify the type of garbage. Because the generative AI model has been pre-trained on a large dataset of garbage images, it is able to identify the type of garbage with high accuracy.

[0166] 4. The server sends the identification result to the robotic means.

[0167] Sorting garbage

[0168] 5. The terminal (robotics system) receives the identification results from the server and begins to act based on that information. Specifically, the robotic arm moves and transports the identified waste to the appropriate sorting zone.

[0169] Display of sorting instructions

[0170] 6. The terminal provides the user with guidance on proper waste separation, using a display and a voice guidance system.

[0171] User Feedback

[0172] 7. Users send feedback to the system through a smartphone app. For example, if a garbage item is misclassified, they can send feedback from the app, such as, "This garbage is not a plastic bottle, but an aluminum can."

[0173] Learning from feedback data

[0174] 8. The server aggregates user feedback data and retrains the generative AI model to continuously improve its classification accuracy, periodically updating the model using online learning techniques.

[0175] Specific examples

[0176] Example 1: Smart trash can for home use

[0177] The device (a camera built into the smart trash can) captures an image of the plastic bottle.

[0178] The terminal transmits this image data to the server.

[0179] The server uses a generative AI model to identify the bottle as a PET bottle and sort it into a separate zone for plastics.

[0180] The terminal opens the plastic lid and guides the user.

[0181] Example 2: Robotics system for waste incineration plants

[0182] The terminal (a camera on the conveyor belt of the waste disposal plant) captures images of the mixed waste.

[0183] The terminal transmits the image data to the server.

[0184] The server uses a generative AI model to identify each type of garbage.

[0185] The server sends each identification result to the robotics system.

[0186] A robotic means (robot arm) transports each waste to the appropriate sorting zone according to the identification results.

[0187] Prompt Sentence Examples

[0188] "What is this garbage?"

[0189] "Please identify the trash in this image."

[0190] "Which sorting zone should this garbage be sorted into?"

[0191] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0192] Step 1:

[0193] The device uses a camera to capture images of the garbage. The device uses a high-resolution camera to capture pixel information. The captured image data is temporarily stored in the camera's memory.

[0194] Input: Physical presence of litter

[0195] Data processing: Image capture using a high-resolution camera

[0196] Output: High-resolution image data

[0197] Step 2:

[0198] The image data acquired by the device is sent to the server in real time using Wi-Fi or other communication protocols (e.g., HTTP or WebSocket).

[0199] Input: High-resolution image data

[0200] Data processing: Converting image data into digital format and sending it

[0201] Output: Image data sent to the server

[0202] Step 3:

[0203] The server inputs the received image data into a generative AI model for data analysis. The generative AI model uses deep learning algorithms such as TensorFlow and PyTorch. The model analyzes the image data and identifies the type of garbage.

[0204] Input: Image data

[0205] Data processing / calculation: Image analysis and garbage type identification using generative AI models

[0206] Output: Classification result data (e.g. "PET bottle")

[0207] Step 4:

[0208] The server transmits the identification results to the robotic means using a lightweight communication protocol such as MQTT, which allows for low-latency communication.

[0209] Input: Classification result data

[0210] Data processing: Convert the identification result data into digital format and send it

[0211] Output: Identification result data sent to robotic means

[0212] Step 5:

[0213] The terminal (robotics system) receives the identification results from the server and starts the waste sorting process based on them. Specifically, the robotic arm moves and transports the identified waste to the appropriate sorting zone. The robotic arm used could be a product from KUKA or Fanuc, for example.

[0214] Input: Classification result data

[0215] Data processing: Controlling robot behavior based on classification results

[0216] Output: Execution of waste sorting work (transfer of waste)

[0217] Step 6:

[0218] The terminal provides users with guidance on how to separate their waste. It uses a display and a voice guidance system to provide specific instructions on how to separate waste. The terminal's display uses a highly visible LCD panel, and voice information is provided through a speaker.

[0219] Input: Classification result data

[0220] Data processing: Creating content for waste sorting guidance

[0221] Output: Display or audio guidance for sorting

[0222] Step 7:

[0223] Users can provide feedback through a smartphone app. For example, if a garbage item is misclassified, they can provide feedback such as, "This garbage is not a plastic bottle, but an aluminum can." The app is developed using React Native and Flutter.

[0224] Input: User feedback information

[0225] Data processing: digitization and transmission of feedback information

[0226] Output: Feedback data sent to the server

[0227] Step 8:

[0228] The server collects the received feedback data and uses it to retrain the generative AI model. The feedback data is added to the model training dataset to improve classification accuracy. The model is periodically updated using online learning techniques.

[0229] Input: User feedback data

[0230] Data processing / calculation: Aggregating and generating feedback information Retraining AI models

[0231] Output: An updated generative AI model

[0232] Through these steps, the system automatically separates waste with high accuracy and efficiency, improving recycling rates.

[0233] (Application example 1)

[0234] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0235] Conventional waste sorting systems are primarily used in households and waste disposal plants, making it difficult to efficiently separate waste generated within the plant. Issues also existed, such as problems with identification accuracy and reduced recycling efficiency due to incorrect sorting. Furthermore, the system's self-learning capabilities were insufficient to improve accuracy, and effective use of user feedback was also required.

[0236] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0237] In this invention, the server includes an image acquisition means, a recognition means using a generative AI model, a robot control means, a communication means, and an interface means. This enables automatic and highly accurate waste sorting within the factory. Specifically, efficient waste sorting is achieved by coordinating image acquisition from a camera device, waste type identification using a generative AI model, and waste transportation by a robot arm. In addition, a self-learning function continuously improves the accuracy of identification, and by incorporating user feedback as data, even more accurate recycling is possible.

[0238] "Image acquisition means" refers to a device or sensor used to acquire image data from an object.

[0239] "Recognition means using generative AI models" refers to a method of using artificial intelligence to analyze acquired image data and identify the type of object.

[0240] "Robot control means" refers to a system or device that controls an automatically operating robot to perform a specific action or task.

[0241] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[0242] "Interface means" refers to devices and programs such as operation panels and software applications that make it easier for users to operate the system.

[0243] "Industrial waste sorting equipment" means equipment for automatically identifying waste generated within a factory and transporting it to the appropriate sorting zone.

[0244] "Artificial intelligence with self-learning capabilities" refers to artificial intelligence that has the ability to continuously improve its recognition accuracy based on acquired data and user feedback.

[0245] This invention relates to a smart factory waste sorting robot system that automatically sorts waste generated in factories and improves recycling efficiency. The system includes an image acquisition means, a recognition means using a generative AI model, a robot control means, a communication means, an interface means, an industrial waste sorting device, and an artificial intelligence with a self-learning function, which work together to improve the efficiency of waste sorting.

[0246] The image acquisition means provided by the server acquires image data of waste using a camera device. Examples of hardware used include a camera device such as the Logitech C920. This camera device is installed on the conveyor belt where waste is mixed in the factory and takes images periodically.

[0247] The server inputs the acquired image data into a recognition means using a generative AI model to identify the type of waste. This generative AI model is built using TensorFlow Keras and performs highly accurate waste identification. The server then sends the identification results to the robot control means, which provides instructions for transporting the waste to the appropriate sorting zone.

[0248] The robot control means provided by the terminal operates the robot arm based on instructions from the server and transports the waste to the appropriate sorting zone. An example of the robot arm used is the UR5 robot arm. The robot control means is also equipped with an industrial waste sorting device, enabling efficient transport of waste.

[0249] The communication means includes the network and communication protocols for transmitting and receiving data between the server, terminal, and other devices, which allows the entire system to work together.

[0250] The interface means includes an operation panel and software applications that allow users to easily operate the system, allowing them to provide feedback to the system and contribute to improving its accuracy. Furthermore, the system has a self-learning function that allows it to continuously improve its classification accuracy based on user feedback.

[0251] As a concrete example, imagine a conveyor belt with mixed waste. A camera on this conveyor belt periodically takes images and sends them to a generative AI model for identification. For example, use the following prompt:

[0252] Input image: (uploaded garbage image)

[0253] ---

[0254] Question: What type of trash is in this image?

[0255] ---

[0256] Example output:

[0257] plastic

[0258] metal

[0259] General waste

[0260] As described above, the present invention enables waste materials in a factory to be separated efficiently and with high accuracy, thereby improving recycling efficiency.

[0261] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0262] Step 1:

[0263] The terminal uses a camera device to capture images of the waste on the conveyor belt. The input is image data from the camera device and the output is the captured image. This image data is sent to the next processing step.

[0264] Step 2:

[0265] The device sends the acquired image data to the server. The input is image data from the camera device, and the output is the image data sent to the server. The server receives this data and analyzes it using a generative AI model.

[0266] Step 3:

[0267] The server inputs the image data into a generative AI model to identify the type of waste. The input is the submitted image data, and the output is the identified waste type (e.g., plastic, metal, general waste). The identification result is sent to the next processing step.

[0268] Step 4:

[0269] The server transmits the identification result to the robot control means, where the input is the identification result regarding the type of waste, and the output is instruction data for the robot control means, including instructions for transporting the waste to the appropriate sorting zone.

[0270] Step 5:

[0271] The terminal controls the robot arm to transport the identified waste to the appropriate sorting zone. The input is instruction data from the server, and the output is the waste transported to the appropriate sorting zone. The robot arm operates based on the identification result.

[0272] Step 6:

[0273] The user operates the system using an interface means and provides feedback as needed. The input is the user's operation and feedback data, and the output is the feedback information sent to the system. This feedback information is collected by the server and used to update the self-learning artificial intelligence.

[0274] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0275] 1. System Overview

[0276] The present invention is a system for automatically sorting waste in homes and waste disposal plants to improve recycling rates. In particular, this system features an emotion engine that recognizes the user's emotions, providing more appropriate feedback and guidance. The system includes image recognition means, generative artificial intelligence means, robotics means, communication means, interface means, and the emotion engine, which work together to make waste sorting more efficient.

[0277] 2. System Configuration

[0278] Image recognition method: Images of garbage are captured using cameras and sensors, and the data is analyzed to identify the type of garbage.

[0279] Generative AI means: Identifies the type of garbage using acquired image data, and improves accuracy through self-learning functions.

[0280] Robotic means: Devices such as robotic arms and conveyor belts that transport identified waste to the appropriate sorting zone.

[0281] Communications: The network and communications protocols used to send and receive data to and from servers and other devices.

[0282] Interface means: An operating panel or software application that facilitates user interaction with the system.

[0283] Emotion engine: Technology that recognizes user emotions and optimizes feedback and guidance based on those emotions.

[0284] 3. Program Processing and Operation

[0285] Garbage image acquisition and recognition

[0286] 1. The device (e.g., a camera mounted on a smart trash can or a garbage truck) captures an image every time garbage is dropped in and sends the image data to a server.

[0287] 2. The server inputs the received image data into a generative AI model to identify the type of garbage.

[0288] 3. The server sends the identification result to the robotic means.

[0289] Sorting garbage

[0290] 4. Based on the identification results from the server, the terminal (robotics system) operates the robotic arm and conveyor belt to transport the waste to the appropriate sorting zone.

[0291] Home and waste incineration plant systems

[0292] 5. The terminal will provide voice guidance and display the type of garbage, and guide the user on the appropriate sorting method.

[0293] 6. Users can also use a smartphone app to take photos of trash and send them to the server, which then notifies the user of the image identification results and sorting instructions.

[0294] Emotion Recognition and Feedback

[0295] 7. The user's emotion data collected on the terminal is sent to the server via communication means.

[0296] 8. The server uses an emotion engine to recognize the user's emotional state and collect it as feedback data.

[0297] 9. The server updates the generative artificial intelligence means based on the emotion data to improve the recognition accuracy and user experience of the entire system.

[0298] 4. Specific Examples

[0299] Example 1: Smart trash can for home use

[0300] The device (a camera built into the smart trash can) captures an image of the plastic bottle.

[0301] The terminal transmits the image data to the server.

[0302] The server uses generative AI to identify it as a plastic bottle and sends instructions to the smart trash can to sort it into a plastic sorting zone.

[0303] The terminal opens the plastic lid based on the instructions and guides the user.

[0304] The device analyzes the user's emotions using an emotion engine, and if stress is felt, it changes the tone of the voice to provide guidance to reassure the user.

[0305] Example 2: Robotics system for waste incineration plants

[0306] The terminal (a camera on the conveyor belt at the waste disposal plant) captures images of the mixed waste.

[0307] The terminal transmits the image data to the server.

[0308] The server uses a generative AI to identify each type of garbage.

[0309] The server sends each identification result to the robotics system.

[0310] A robotic means (robot arm) transports each waste to the appropriate sorting zone according to the identification results.

[0311] The server uses an emotion engine to collect workers' emotional data and monitor their fatigue and stress levels.

[0312] 5. Feedback and self-learning

[0313] Users can send feedback through the smart trash can or smartphone app, and if the sorting is incorrect, the user's feedback will be reflected in the system.

[0314] The server collects feedback and emotion data and updates the generative AI model, which continuously improves its classification accuracy.

[0315] In this way, the system of the present invention not only improves the efficiency of waste sorting at homes and waste disposal plants and achieves highly accurate recycling, but also provides a comfortable experience that takes the user's emotions into consideration.

[0316] The processing flow will be explained below.

[0317] Processing flow for smart household trash cans

[0318] Step 1: Image capture of the garbage

[0319] Operation details:

[0320] The device (a camera built into the smart trash can) takes an image every time trash is dropped in.

[0321] Step 2: Sending images

[0322] Operation details:

[0323] The terminal transmits the captured image data to the server.

[0324] Step 3: Image Identification (using generative AI)

[0325] Operation details:

[0326] The server inputs the received image data into a generative AI model to identify the type of garbage.

[0327] Step 4: Notification of identification results

[0328] Operation details:

[0329] The server transmits the identification result to the terminal.

[0330] Step 5: Providing guidance on waste sorting

[0331] Operation details:

[0332] The terminal opens the lid of the appropriate sorting zone based on the type of waste identified.

[0333] The terminal guides the user on how to separate the waste by voice guidance or display on the screen.

[0334] Step 6: Collect emotion data

[0335] Operation details:

[0336] The terminal uses an emotion engine to collect the user's voice tone and movement data.

[0337] Step 7: Sending Emotion Data

[0338] Operation details:

[0339] The terminal transmits the collected emotion data to the server.

[0340] Step 8: Analyze the sentiment data

[0341] Operation details:

[0342] The server uses an emotion engine to analyze the user's emotional state and generate feedback based thereon.

[0343] Step 9: View your feedback

[0344] Operation details:

[0345] The server transmits feedback based on the analysis results to the terminal.

[0346] The terminal provides the user with a steady tone and thoughtful guidance.

[0347] Processing flow in a robotics system for waste incineration plants

[0348] Step 1: Image capture of the garbage

[0349] Operation details:

[0350] The terminal (a camera on the conveyor belt at the waste disposal plant) takes images of the waste as it flows by.

[0351] Step 2: Sending images

[0352] Operation details:

[0353] The terminal transmits the captured image data to the server.

[0354] Step 3: Image Identification (using generative AI)

[0355] Operation details:

[0356] The server inputs the received image data into a generative AI model to identify each type of garbage.

[0357] Step 4: Notification of identification results

[0358] Operation details:

[0359] The server sends each identification result to the robotics system.

[0360] Step 5: Sorting your trash

[0361] Operation details:

[0362] The terminal (robot arm) transports each piece of garbage to the appropriate sorting zone based on the identification results from the server.

[0363] Step 6: Collect worker emotion data

[0364] Operation details:

[0365] The device uses an emotion engine to analyze the worker's movements and voice and collect emotional data.

[0366] Step 7: Sending Emotion Data

[0367] Operation details:

[0368] The terminal transmits the collected emotion data to the server.

[0369] Step 8: Worker condition monitoring

[0370] Operation details:

[0371] The server uses an emotion engine to monitor workers' stress levels and fatigue and generate feedback.

[0372] Step 9: Provide monitoring results

[0373] Operation details:

[0374] The server sends the monitoring results to the terminal and notifies the workers and managers.

[0375] Processing flow for smartphone app integration

[0376] Step 1: Photograph the litter

[0377] Operation details:

[0378] Users take photos of the trash using a smartphone app.

[0379] Step 2: Send your photos

[0380] Operation details:

[0381] The terminal transmits the photographed photo data to the server.

[0382] Step 3: Image Identification (using generative AI)

[0383] Operation details:

[0384] The server inputs the received photo data into a generative AI model to identify the type of litter.

[0385] Step 4: Propose a sorting method

[0386] Operation details:

[0387] Based on the identification results, the server sends the appropriate sorting method to the terminal (smartphone app).

[0388] The terminal displays the sorting method to the user.

[0389] Step 5: Collecting sentiment data

[0390] Operation details:

[0391] The device analyzes the user's voice and actions using an emotion engine to collect emotion data.

[0392] Step 6: Sending Emotion Data

[0393] Operation details:

[0394] The terminal transmits the collected emotion data to the server.

[0395] Step 7: Analyze the sentiment data

[0396] Operation details:

[0397] The server uses an emotion engine to analyze the user's emotional state and generate feedback.

[0398] Step 8: Viewing feedback

[0399] Operation details:

[0400] The server sends optimized feedback based on the emotion data to the terminal.

[0401] The terminal displays a thoughtful guide to the user.

[0402] Feedback and self-learning process flow

[0403] Step 1: Submit your feedback

[0404] Operation details:

[0405] Users can provide feedback through the smart trash can or through a smartphone app.

[0406] Step 2: Receive and analyze feedback

[0407] Operation details:

[0408] The server analyzes the received feedback data.

[0409] Step 3: Update the AI ​​model

[0410] Operation details:

[0411] The server updates the generative AI model based on the feedback data.

[0412] Step 4: Apply the model

[0413] Operation details:

[0414] The server will use the updated AI model and apply it to future image recognition processes.

[0415] Step 5: Learning with emotion data

[0416] Operation details:

[0417] The server uses the data collected by the emotion engine to self-train the generative AI and provide feedback according to emotions.

[0418] In this way, the system of the present invention not only improves the efficiency of waste sorting at homes and waste disposal plants and achieves highly accurate recycling, but also provides a comfortable experience that takes the user's emotions into consideration.

[0419] Example 2

[0420] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0421] Conventional waste sorting systems have limitations in the accuracy and efficiency of sorting, and also have the problem of being unable to take user emotions and feedback into account. Furthermore, waste sorting systems for households and waste disposal plants have also been plagued by issues of incorrect sorting and low system learning accuracy. This has led to a demand for improved recycling rates and user experience.

[0422] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0423] In this invention, the server includes image recognition means, generative artificial intelligence means, robotics means, communication means, interface means, emotion recognition means, user feedback processing means, and self-learning means, which not only enables accurate and efficient waste sorting but also enables feedback that takes user emotions into account and continuous system learning and improvement.

[0424] "Image recognition means" refers to devices and technologies that use cameras or sensors to capture images of garbage and analyze the data to identify the type of garbage.

[0425] "Generative artificial intelligence means" refers to an artificial intelligence system that analyzes acquired image data, uses self-learning functions to identify the type of garbage, and improves identification accuracy.

[0426] "Robotics means" refers to devices such as robotic arms and conveyor belts used to transport identified waste to the appropriate sorting zone.

[0427] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[0428] "Interface means" refers to an operation panel or software application that facilitates user interaction with the system.

[0429] "Emotion recognition means" refers to technology and devices that recognize a user's emotions and optimize feedback and guidance based on those emotions.

[0430] "User feedback processing means" refers to processes and techniques for collecting feedback from users and using it to improve the system.

[0431] "Self-learning tools" refers to learning algorithms and techniques that use collected data to continuously improve the overall system's recognition accuracy and user experience.

[0432] MODE FOR CARRYING OUT THE INVENTION

[0433] System Overview

[0434] The present invention is a system that automatically separates waste in homes and waste disposal plants to improve recycling rates. In particular, the system features an emotion engine that recognizes the user's emotions, providing more appropriate feedback and guidance. The system includes image recognition means, generative artificial intelligence means, robotics means, communication means, interface means, emotion recognition means, user feedback processing means, and self-learning means, which work together to streamline waste separation.

[0435] Hardware and Software Configuration

[0436] The specific hardware and software used in the system are as follows:

[0437] 1. Devices (e.g., smart trash cans and cameras mounted on garbage trucks)

[0438] Hardware: High-resolution camera (e.g. Logitech C920)

[0439] Communication method: Wi-Fi and LTE module

[0440] Interface: Operation panel, display, microphone (e.g., Amazon Polly for voice guidance)

[0441] Emotion Recognition Technology: Microsoft's Emotion API

[0442] 2. Server

[0443] Hardware: High-performance servers (e.g. AWS EC2)

[0444] Software: Generative AI models (e.g., TensorFlow, PyTorch)

[0445] System Operation and Processing

[0446] The operation of this system is realized as follows.

[0447] The image recognition means uses a camera to capture an image of the garbage and transmits the data to a server.

[0448] The artificial intelligence generating means analyzes the image data received by the server and identifies the type of waste using a self-learning function.

[0449] The identification result is transmitted to the robotics means through the communication means.

[0450] The robotic means operates robotic arms and conveyor belts to transport identified waste to the appropriate sorting zone.

[0451] The interface means provides guidance to the user by voice or display.

[0452] The emotion recognition means recognizes the user's emotions and provides optimal guidance based on the feedback.

[0453] The user feedback processing means collects feedback from users and reflects it in improving the system.

[0454] The self-learning mechanism continuously updates the generative AI model based on collected data, improving recognition accuracy and user experience.

[0455] Specific operation example

[0456] Example of a smart trash can for home use

[0457] 1. The device (the camera built into the smart trash can) captures an image of the plastic bottle.

[0458] 2. The terminal sends the image data to the server.

[0459] 3. The server uses a generative AI model to identify it as a plastic bottle and sends instructions to the smart trash can to sort it into a plastic sorting zone.

[0460] 4. The terminal opens the plastic lid based on the instructions and guides the user.

[0461] 5. The device analyzes the user's emotions using an emotion engine, and if stress is sensed, it changes the voice tone to provide guidance to reassure the user.

[0462] Example of a robotics system for an incineration plant

[0463] 1. The terminal (a camera on the conveyor belt at the waste disposal plant) captures images of mixed waste.

[0464] 2. The terminal sends the image data to the server.

[0465] 3. The server uses a generative AI model to identify each type of garbage.

[0466] 4. The server sends each identification result to the robotics system.

[0467] 5. The robotic means (robot arm) transports each waste to the appropriate sorting zone according to the identification results.

[0468] 6. The server uses an emotion engine to collect workers' emotional data and monitor their fatigue and stress levels.

[0469] Prompt Sentence Examples

[0470] Prompt for sorting waste with a smart trash can for home:

[0471] Describe what happens to a plastic bottle after it's dropped into a smart trash can. The system includes a camera, a server, robotics tools, and an emotion engine.

[0472] Prompt for sorting garbage at an incineration plant:

[0473] Describe what happens to the mixed waste after it's placed on the conveyor belt. The system includes a camera, a server, robotics tools, and an emotion engine.

[0474] In this way, the system of the present invention not only improves the efficiency of waste sorting at homes and waste disposal plants and achieves high-precision recycling, but also provides a comfortable experience that takes user emotions into consideration.

[0475] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0476] Step 1:

[0477] The device (the built-in camera of the smart trash can) automatically captures images when trash is dropped in. The input is a physical image of the trash, which is captured by the camera and converted into image data in JPEG format. The camera uses a high-resolution camera (e.g., Logitech C920) to adjust the exposure and focus in real time.

[0478] Step 2:

[0479] The device sends the acquired image data to the server via the Wi-Fi module. The input is the JPEG image data acquired in step 1, which is encrypted and sent. The data is sent securely using TLS (Transport Layer Security).

[0480] Step 3:

[0481] The server inputs the received image data into the generative AI model. The input is JPEG image data, which is input to a virtual machine running TensorFlow on the server. The generative AI model analyzes the image data and performs data calculations to identify the type of garbage. The output is the result of identifying the type of garbage.

[0482] Step 4:

[0483] The server converts the identification results into JSON format and sends them back to the device via a communication method. The input is the identification results from the TensorFlow model, which are serialized into JSON format. The output is the JSON-formatted identification results, which are sent via the secure WebSocket protocol.

[0484] Step 5:

[0485] The terminal (robotics system) operates the robotic arm and conveyor belt based on the identification results. The input is the identification results in JSON format from the server, which is analyzed to generate specific operation instructions. For example, it operates an ABB IRB 6700 robotic arm to perform a specific operation, such as transporting aluminum cans to a metal sorting zone. The output is that the waste is transported to the appropriate sorting zone.

[0486] Step 6:

[0487] The device provides the user with the sorting results and instructions through voice guidance and a display. The inputs are the JSON formatted identification results and text for the voice guidance, which are converted into speech using speech synthesis technology (e.g., Amazon Polly). The output is a message saying, "This garbage is a plastic bottle. Please use a plastic recycling lid."

[0488] Step 7:

[0489] The device collects the user's emotional data and sends it to the server. The input includes facial expressions and voices while the user is performing the classification activity, and data is collected using an facial recognition camera and microphone. This data is analyzed using Microsoft's Emotion API and converted into emotional data. The output is the analyzed emotional data, which is sent to the server via secure communication.

[0490] Step 8:

[0491] The server updates the generative AI model based on emotion data and user feedback. The inputs are emotion data obtained from the Emotion API and user feedback data, which are analyzed to adjust the parameters of the generative AI model. The data is then processed to retrain the model's neural network and improve its recognition accuracy. The output is an updated generative AI model, which improves recognition accuracy and the user experience from the next time onwards.

[0492] (Application example 2)

[0493] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0494] Traditionally, parts sorting and quality inspection in factories have been largely manual and inefficient. Furthermore, workers' stress and emotional states have not been properly managed, leading to a deterioration in the working environment. This has led to reduced production efficiency and increased likelihood of work errors, affecting overall work efficiency. To solve these problems, a system is needed that can automatically identify and sort parts while simultaneously managing the emotional states of workers.

[0495] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0496] In this invention, the server includes image recognition means, generative artificial intelligence means, robotics means, communication means, interface means, emotion recognition means, and emotion-based feedback means, thereby improving the efficiency of automatic part identification and sorting, and enabling real-time monitoring of the emotional state of workers and appropriate feedback.

[0497] An "image recognition means" is a device that uses a camera or sensor to capture an image of an object, analyzes the data, and identifies the type and condition of the object.

[0498] The "generative artificial intelligence means" is an artificial intelligence system that identifies the target object based on the acquired image data and improves the accuracy of identification through its self-learning function.

[0499] "Robotics means" refers to devices such as robotic arms and conveyor belts that transport identified objects to the appropriate location.

[0500] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[0501] "Interface means" refers to an operation panel or software application that facilitates user interaction with the system.

[0502] "Emotion recognition means" refers to a device or software that analyzes a user's facial image to determine the user's emotional state.

[0503] The "means for providing feedback based on emotions" refers to a device or software that provides appropriate guidance or feedback to the user based on the results of emotion recognition.

[0504] This invention is a system for improving the efficiency of parts sorting and quality inspection in factories. It also monitors the emotional state of workers in real time and manages their stress. The specific system configuration, usage, and operational procedures are described below.

[0505] 1. System Configuration

[0506] The system includes the following main elements:

[0507] Image recognition means: The camera in the smart glasses captures an image of the part.

[0508] Generative AI means: Analyzes acquired image data and identifies the type and quality of parts. For example, it has the ability to determine whether a product is good or defective.

[0509] Robotic means: Robotic arms and conveyor belts to transport identified parts to the appropriate location.

[0510] Communications: The network and communications protocols used to send and receive data to and from servers and other devices.

[0511] Interface means: An operation panel or software application that allows the worker to interact with the system. This can be the display or voice guidance on the smart glasses.

[0512] Emotion recognition means: Software that captures facial images of workers using the camera in the smart glasses and analyzes their emotional state.

[0513] Means of providing emotion-based feedback: Analyze the emotional state and provide feedback based on stress, fatigue, etc. via audio guidance, etc.

[0514] 2. Program Processing

[0515] Image Acquisition and Recognition

[0516] The device (the camera in the smart glasses) captures an image of the part. Image recognition means is used to analyze the image data and identify the type and condition (good or bad) of the part.

[0517] The server sends the image data to a generative AI model for part identification, which automatically identifies the part type and quality.

[0518] Emotion Recognition and Feedback

[0519] The device (the camera in the smart glasses) captures facial images of the worker, which are then analyzed by emotion recognition means to determine the worker's emotional state (especially their stress level).

[0520] Based on the results of emotion recognition, the server analyzes the data using generative artificial intelligence and provides appropriate guidance to the worker through emotion-based feedback, such as "take a break" if the worker is feeling high stress.

[0521] As a concrete example, imagine a scene where a worker is wearing smart glasses and working on a factory production line. Every time a part comes through, the camera in the smart glasses takes an image of the part, and the AI ​​immediately identifies whether it is a good or defective part. If a defective part is detected, a voice message will be played saying, "A defective part has been detected. Please remove it." The system will also analyze the worker's facial expression, and if the worker's stress level is high, a voice message will be played saying, "You are under a lot of stress. Please take a short break."

[0522] 3. Example prompts

[0523] Input image: An image of a part on a factory production line

[0524] Task: Identify whether the part is good or bad.

[0525] Input image: Face image of a worker

[0526] Task: Identify emotions and stress levels from workers' facial expressions.

[0527] This system will significantly improve the efficiency of automatic identification and sorting of parts in factories, and by monitoring the emotional state of workers and providing appropriate feedback, it will improve the working environment and increase production efficiency.

[0528] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0529] Step 1:

[0530] The device uses the camera in the smart glasses to capture images of parts on the factory production line. The input is image data of the parts, and the output is an image file.

[0531] Step 2:

[0532] The terminal transmits the acquired image data to the server using a communication means. The input is the image file, and the output is the result of the data transmission to the server.

[0533] Step 3:

[0534] The server inputs the received image data into the artificial intelligence generating means to identify the type and quality of the part. The input is the image file, and the output is the label of the part's identification result (e.g., good, defective).

[0535] Step 4:

[0536] The server sends the identification results to the robotics means and issues instructions for automatic sorting. The input is the label of the part identification result, and the output is operation instructions for the robot arm and conveyor belt.

[0537] Step 5:

[0538] The terminal acquires the facial image of the worker using the camera in the smart glasses. The input is the facial image data, and the output is a facial image file.

[0539] Step 6:

[0540] The terminal transmits the facial image data to the server using a communication means. The input is a facial image file, and the output is the result of the data transmission to the server.

[0541] Step 7:

[0542] The server inputs the received facial image data into the emotion recognition means to determine the worker's emotional state and stress level. The input is a facial image file, and the output is a label of the emotional state and a numerical value of the stress level.

[0543] Step 8:

[0544] The server generates a feedback message based on the emotional state and stress level using a generation artificial intelligence means and transmits it to the terminal, where the input is the label of the emotional state and the numerical value of the stress level, and the output is the feedback message.

[0545] Step 9:

[0546] The terminal provides the worker with a feedback message through an interface means such as voice guidance or display, etc. The input is the feedback message, and the output is the voice or display.

[0547] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0548] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0549] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0550] [Second embodiment]

[0551] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0552] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0553] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0554] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0555] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0556] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0557] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0558] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0559] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0560] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0561] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0562] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0563] 1. System Overview

[0564] The present invention is a system for automatically separating waste in homes and waste disposal plants, thereby improving recycling rates. The system includes image recognition means, generative artificial intelligence means, robotics means, communication means, and interface means, which work together to make waste separation more efficient.

[0565] 2. System Configuration

[0566] Image recognition method: Images of garbage are captured using cameras and sensors, and the data is analyzed to identify the type of garbage.

[0567] Generative AI means: Identifies the type of garbage using acquired image data, and improves accuracy through self-learning functions.

[0568] Robotic means: Devices such as robotic arms and conveyor belts that transport identified waste to the appropriate sorting zone.

[0569] Communications: The network and communications protocols used to send and receive data to and from servers and other devices.

[0570] Interface means: An operating panel or software application that facilitates user interaction with the system.

[0571] 3. Program Processing and Operation

[0572] Garbage image acquisition and recognition

[0573] 1. The device (e.g., a camera mounted on a smart trash can or a garbage truck) captures an image every time garbage is dropped in and sends the image data to a server.

[0574] 2. The server inputs the received image data into a generative AI model to identify the type of garbage.

[0575] 3. The server sends the identification result to the robotic means.

[0576] Sorting garbage

[0577] 4. Based on the identification results from the server, the terminal (robotics system) operates the robotic arm and conveyor belt to transport the waste to the appropriate sorting zone.

[0578] Home and waste incineration plant systems

[0579] 5. The terminal will provide voice guidance and display the type of garbage, and guide the user on the appropriate sorting method.

[0580] 6. Users can also use a smartphone app to take photos of trash and send them to the server, which then notifies the user of the image identification results and sorting instructions.

[0581] 4. Specific Examples

[0582] Example 1: Smart trash can for home use

[0583] The device (a camera built into the smart trash can) captures an image of the plastic bottle.

[0584] The terminal transmits the image data to the server.

[0585] The server uses generative AI to identify it as a plastic bottle and sends instructions to the smart trash can to sort it into a plastic sorting zone.

[0586] The terminal opens the plastic lid based on the instructions and guides the user.

[0587] Example 2: Robotics system for waste incineration plants

[0588] The terminal (a camera on the conveyor belt at the waste disposal plant) captures images of the mixed waste.

[0589] The terminal transmits the image data to the server.

[0590] The server uses a generative AI to identify each type of garbage.

[0591] The server sends each identification result to the robotics system.

[0592] A robotic means (robot arm) transports each waste to the appropriate sorting zone according to the identification results.

[0593] 5. Feedback and self-study

[0594] Users can send feedback through the smart trash can or smartphone app, and if the sorting is incorrect, the user's feedback will be reflected in the system.

[0595] The server collects feedback data and updates the generative AI model, which continuously improves its classification accuracy.

[0596] In this way, the system of the present invention improves the efficiency of waste sorting at homes and waste disposal plants, and realizes highly accurate recycling.

[0597] The processing flow will be explained below.

[0598] Processing flow for smart household trash cans

[0599] Step 1: Image capture of the garbage

[0600] Operation details:

[0601] The device (a camera built into the smart trash can) takes an image every time trash is dropped in.

[0602] Step 2: Sending images

[0603] Operation details:

[0604] The terminal transmits the captured image data to the server.

[0605] Step 3: Image Identification (using generative AI)

[0606] Operation details:

[0607] The server inputs the received image data into a generative AI model to identify the type of garbage.

[0608] Step 4: Notification of identification results

[0609] Operation details:

[0610] The server transmits the identification result to the terminal.

[0611] Step 5: Providing guidance on waste sorting

[0612] Operation details:

[0613] The terminal opens the lid of the appropriate sorting zone based on the type of waste identified.

[0614] The terminal guides the user on how to separate the waste by voice guidance or display on the screen.

[0615] Processing flow in a robotics system for waste incineration plants

[0616] Step 1: Image capture of the garbage

[0617] Operation details:

[0618] The terminal (a camera on the conveyor belt at the waste disposal plant) takes images of the waste as it flows by.

[0619] Step 2: Sending images

[0620] Operation details:

[0621] The terminal transmits the captured image data to the server.

[0622] Step 3: Image Identification (using generative AI)

[0623] Operation details:

[0624] The server inputs the received image data into a generative AI model to identify each type of garbage.

[0625] Step 4: Notification of identification results

[0626] Operation details:

[0627] The server sends each identification result to the robotics system.

[0628] Step 5: Sorting your trash

[0629] Operation details:

[0630] The terminal (robot arm) transports each piece of garbage to the appropriate sorting zone based on the identification results from the server.

[0631] Processing flow for smartphone app integration

[0632] Step 1: Photograph the litter

[0633] Operation details:

[0634] Users take photos of the trash using a smartphone app.

[0635] Step 2: Send your photos

[0636] Operation details:

[0637] The terminal transmits the photographed photo data to the server.

[0638] Step 3: Image Identification (using generative AI)

[0639] Operation details:

[0640] The server inputs the received image data into a generative AI model to identify the type of garbage.

[0641] Step 4: Propose a sorting method

[0642] Operation details:

[0643] Based on the identification results, the server sends the appropriate sorting method to the terminal (smartphone app).

[0644] The terminal displays the sorting method to the user.

[0645] Feedback and self-learning process flow

[0646] Step 1: Submit your feedback

[0647] Operation details:

[0648] Users can provide feedback on their sorting results via a smartphone app or smart trash can.

[0649] Step 2: Receive and analyze feedback

[0650] Operation details:

[0651] The server analyzes the received feedback data.

[0652] Step 3: Update the AI ​​model

[0653] Operation details:

[0654] The server updates the generative AI model based on the feedback data.

[0655] Step 4: Apply the model

[0656] Operation details:

[0657] The server will use the updated AI model and apply it to future image recognition processes.

[0658] Through these processing steps, the sorting accuracy of the entire system will improve, aiming to increase recycling rates and realize a sustainable society.

[0659] Example 1

[0660] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0661] Conventional waste sorting at homes and waste disposal plants relies on manual labor or simple machines, resulting in low sorting accuracy and limiting the improvement of recycling rates. Manual sorting is also labor-intensive and time-consuming, and has many issues with efficiency. Furthermore, when incorrect sorting occurs, it is difficult to incorporate user feedback into the system, and the system lacks a self-learning function. There is a need for a system that can solve these problems and achieve high-precision, efficient waste sorting.

[0662] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0663] In this invention, the server includes a means for transmitting image data from the terminal to the server, a means for analyzing image data in the server, and a means for transmitting identification results from the server to the robotics means. This enables highly accurate identification of waste types. The server also includes a means for receiving feedback data from users and a means for self-learning and updating the generative artificial intelligence model, allowing for continuous improvement in identification accuracy through feedback. Furthermore, the robotics means automatically transports waste to the sorting zone, significantly improving work efficiency. This system can be used for both home and waste disposal plants, contributing to increased recycling rates.

[0664] "Image recognition means" refers to a device or software that analyzes acquired image data and identifies the type of garbage.

[0665] "Generative artificial intelligence means" refers to an AI model and related system that uses acquired image data to identify the type of garbage and improves identification accuracy through self-learning functions.

[0666] "Robotics means" refers to devices such as robotic arms and conveyor belts that transport identified waste to the appropriate sorting zone.

[0667] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[0668] "Interface means" refers to an operation panel or software application that facilitates user interaction with the system.

[0669] The "means for transmitting image data from the terminal to the server" is a function for transmitting image data captured by the terminal to the server in real time.

[0670] "Image data analysis means on the server" is a function that inputs the image data received by the server into a generative AI model, analyzes it, and identifies the type of garbage.

[0671] The "means for transmitting identification results from the server to the robotics means" is a function for transmitting the identification results analyzed by the server to the robotics means.

[0672] The "means for receiving feedback data from users" is a function that receives feedback data provided by users via smartphone apps or the like on the server.

[0673] The "self-learning update means for the generative AI model" is a function that updates the learning of the generative AI model using received feedback data to improve the recognition accuracy.

[0674] System Overview

[0675] The present invention is a system for automatically separating waste at homes and waste disposal plants to improve recycling rates. This system operates by linking together image recognition means, generative artificial intelligence means, robotics means, communication means, and interface means.

[0676] Hardware and software used

[0677] Image recognition method: Images of garbage are captured using cameras or sensors, and the data is analyzed. Specifically, a high-resolution camera (e.g., Sony's IMX577 sensor) is used.

[0678] Generative AI method: The acquired image data is input into a generative AI model (e.g., a deep learning model using TensorFlow or PyTorch) to identify the type of garbage.

[0679] Robotic means: using robotic arms or conveyor belts that transport identified waste to the appropriate sorting zone. Examples include robotic arms from KUKA and Fanuc.

[0680] Communication method: Uses a network (e.g., Wi-Fi module and communication protocols such as HTTP, WebSocket, and MQTT) to send and receive data to and from servers and other devices.

[0681] Interface means: An operation panel or software application that makes it easy for the user to interact with the system. Examples include Android and iOS applications.

[0682] Image acquisition and identification of debris

[0683] 1. The device uses cameras installed in smart trash bins or on conveyor belts at waste disposal plants to capture images every time trash is dropped in.

[0684] 2. The device transmits the acquired image data to the server in real time.

[0685] 3. The server inputs the received image data into a generative AI model to identify the type of garbage. Because the generative AI model has been pre-trained on a large dataset of garbage images, it is able to identify the type of garbage with high accuracy.

[0686] 4. The server sends the identification result to the robotic means.

[0687] Sorting garbage

[0688] 5. The terminal (robotics system) receives the identification results from the server and begins to act based on that information. Specifically, the robotic arm moves and transports the identified waste to the appropriate sorting zone.

[0689] Display of sorting instructions

[0690] 6. The terminal provides the user with guidance on proper waste separation, using a display and a voice guidance system.

[0691] User Feedback

[0692] 7. Users send feedback to the system through a smartphone app. For example, if a garbage item is misclassified, they can send feedback from the app, such as, "This garbage is not a plastic bottle, but an aluminum can."

[0693] Learning from feedback data

[0694] 8. The server aggregates user feedback data and retrains the generative AI model to continuously improve its classification accuracy, periodically updating the model using online learning techniques.

[0695] Specific examples

[0696] Example 1: Smart trash can for home use

[0697] The device (a camera built into the smart trash can) captures an image of the plastic bottle.

[0698] The terminal transmits this image data to the server.

[0699] The server uses a generative AI model to identify the bottle as a PET bottle and sort it into a separate zone for plastics.

[0700] The terminal opens the plastic lid and guides the user.

[0701] Example 2: Robotics system for waste incineration plants

[0702] The terminal (a camera on the conveyor belt of the waste disposal plant) captures images of the mixed waste.

[0703] The terminal transmits the image data to the server.

[0704] The server uses a generative AI model to identify each type of garbage.

[0705] The server sends each identification result to the robotics system.

[0706] A robotic means (robot arm) transports each waste to the appropriate sorting zone according to the identification results.

[0707] Prompt Sentence Examples

[0708] "What is this garbage?"

[0709] "Please identify the trash in this image."

[0710] "Which sorting zone should this garbage be sorted into?"

[0711] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0712] Step 1:

[0713] The device uses a camera to capture images of the garbage. The device uses a high-resolution camera to capture pixel information. The captured image data is temporarily stored in the camera's memory.

[0714] Input: Physical presence of litter

[0715] Data processing: Image capture using a high-resolution camera

[0716] Output: High-resolution image data

[0717] Step 2:

[0718] The image data acquired by the device is sent to the server in real time using Wi-Fi or other communication protocols (e.g., HTTP or WebSocket).

[0719] Input: High-resolution image data

[0720] Data processing: Converting image data into digital format and sending it

[0721] Output: Image data sent to the server

[0722] Step 3:

[0723] The server inputs the received image data into a generative AI model for data analysis. The generative AI model uses deep learning algorithms such as TensorFlow and PyTorch. The model analyzes the image data and identifies the type of garbage.

[0724] Input: Image data

[0725] Data processing / calculation: Image analysis and garbage type identification using generative AI models

[0726] Output: Classification result data (e.g. "PET bottle")

[0727] Step 4:

[0728] The server transmits the identification results to the robotic means using a lightweight communication protocol such as MQTT, which allows for low-latency communication.

[0729] Input: Classification result data

[0730] Data processing: Convert the identification result data into digital format and send it

[0731] Output: Identification result data sent to robotic means

[0732] Step 5:

[0733] The terminal (robotics system) receives the identification results from the server and starts the waste sorting process based on them. Specifically, the robotic arm moves and transports the identified waste to the appropriate sorting zone. The robotic arm used could be a product from KUKA or Fanuc, for example.

[0734] Input: Classification result data

[0735] Data processing: Controlling robot behavior based on classification results

[0736] Output: Execution of waste sorting work (transfer of waste)

[0737] Step 6:

[0738] The terminal provides users with guidance on how to separate their waste. It uses a display and a voice guidance system to provide specific instructions on how to separate waste. The terminal's display uses a highly visible LCD panel, and voice information is provided through a speaker.

[0739] Input: Classification result data

[0740] Data processing: Creating content for waste sorting guidance

[0741] Output: Display or audio guidance for sorting

[0742] Step 7:

[0743] Users can provide feedback through a smartphone app. For example, if a garbage item is misclassified, they can provide feedback such as, "This garbage is not a plastic bottle, but an aluminum can." The app is developed using React Native and Flutter.

[0744] Input: User feedback information

[0745] Data processing: digitization and transmission of feedback information

[0746] Output: Feedback data sent to the server

[0747] Step 8:

[0748] The server collects the received feedback data and uses it to retrain the generative AI model. The feedback data is added to the model training dataset to improve classification accuracy. The model is periodically updated using online learning techniques.

[0749] Input: User feedback data

[0750] Data processing / calculation: Aggregating and generating feedback information Retraining AI models

[0751] Output: An updated generative AI model

[0752] Through these steps, the system automatically separates waste with high accuracy and efficiency, improving recycling rates.

[0753] (Application example 1)

[0754] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0755] Conventional waste sorting systems are primarily used in households and waste disposal plants, making it difficult to efficiently separate waste generated within the plant. Issues also existed, such as problems with identification accuracy and reduced recycling efficiency due to incorrect sorting. Furthermore, the system's self-learning capabilities were insufficient to improve accuracy, and effective use of user feedback was also required.

[0756] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0757] In this invention, the server includes an image acquisition means, a recognition means using a generative AI model, a robot control means, a communication means, and an interface means. This enables automatic and highly accurate waste sorting within the factory. Specifically, efficient waste sorting is achieved by coordinating image acquisition from a camera device, waste type identification using a generative AI model, and waste transportation by a robot arm. In addition, a self-learning function continuously improves the accuracy of identification, and by incorporating user feedback as data, even more accurate recycling is possible.

[0758] "Image acquisition means" refers to a device or sensor used to acquire image data from an object.

[0759] "Recognition means using generative AI models" refers to a method of using artificial intelligence to analyze acquired image data and identify the type of object.

[0760] "Robot control means" refers to a system or device that controls an automatically operating robot to perform a specific action or task.

[0761] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[0762] "Interface means" refers to devices and programs such as operation panels and software applications that make it easier for users to operate the system.

[0763] "Industrial waste sorting equipment" means equipment for automatically identifying waste generated within a factory and transporting it to the appropriate sorting zone.

[0764] "Artificial intelligence with self-learning capabilities" refers to artificial intelligence that has the ability to continuously improve its recognition accuracy based on acquired data and user feedback.

[0765] This invention relates to a smart factory waste sorting robot system that automatically sorts waste generated in factories and improves recycling efficiency. The system includes an image acquisition means, a recognition means using a generative AI model, a robot control means, a communication means, an interface means, an industrial waste sorting device, and an artificial intelligence with a self-learning function, which work together to improve the efficiency of waste sorting.

[0766] The image acquisition means provided by the server acquires image data of waste using a camera device. Examples of hardware used include a camera device such as the Logitech C920. This camera device is installed on the conveyor belt where waste is mixed in the factory and takes images periodically.

[0767] The server inputs the acquired image data into a recognition means using a generative AI model to identify the type of waste. This generative AI model is built using TensorFlow Keras and performs highly accurate waste identification. The server then sends the identification results to the robot control means, which provides instructions for transporting the waste to the appropriate sorting zone.

[0768] The robot control means provided by the terminal operates the robot arm based on instructions from the server and transports the waste to the appropriate sorting zone. An example of the robot arm used is the UR5 robot arm. The robot control means is also equipped with an industrial waste sorting device, enabling efficient transport of waste.

[0769] The communication means includes the network and communication protocols for transmitting and receiving data between the server, terminal, and other devices, which allows the entire system to work together.

[0770] The interface means includes an operation panel and software applications that allow users to easily operate the system, allowing them to provide feedback to the system and contribute to improving its accuracy. Furthermore, the system has a self-learning function that allows it to continuously improve its classification accuracy based on user feedback.

[0771] As a concrete example, imagine a conveyor belt with mixed waste. A camera on this conveyor belt periodically takes images and sends them to a generative AI model for identification. For example, use the following prompt:

[0772] Input image: (uploaded garbage image)

[0773] ---

[0774] Question: What type of trash is in this image?

[0775] ---

[0776] Example output:

[0777] plastic

[0778] metal

[0779] General waste

[0780] As described above, the present invention enables waste materials in a factory to be separated efficiently and with high accuracy, thereby improving recycling efficiency.

[0781] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0782] Step 1:

[0783] The terminal uses a camera device to capture images of the waste on the conveyor belt. The input is image data from the camera device and the output is the captured image. This image data is sent to the next processing step.

[0784] Step 2:

[0785] The device sends the acquired image data to the server. The input is image data from the camera device, and the output is the image data sent to the server. The server receives this data and analyzes it using a generative AI model.

[0786] Step 3:

[0787] The server inputs the image data into a generative AI model to identify the type of waste. The input is the submitted image data, and the output is the identified waste type (e.g., plastic, metal, general waste). The identification result is sent to the next processing step.

[0788] Step 4:

[0789] The server transmits the identification result to the robot control means, where the input is the identification result regarding the type of waste, and the output is instruction data for the robot control means, including instructions for transporting the waste to the appropriate sorting zone.

[0790] Step 5:

[0791] The terminal controls the robot arm to transport the identified waste to the appropriate sorting zone. The input is instruction data from the server, and the output is the waste transported to the appropriate sorting zone. The robot arm operates based on the identification result.

[0792] Step 6:

[0793] The user operates the system using an interface means and provides feedback as needed. The input is the user's operation and feedback data, and the output is the feedback information sent to the system. This feedback information is collected by the server and used to update the self-learning artificial intelligence.

[0794] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0795] 1. System Overview

[0796] The present invention is a system for automatically sorting waste in homes and waste disposal plants to improve recycling rates. In particular, this system features an emotion engine that recognizes the user's emotions, providing more appropriate feedback and guidance. The system includes image recognition means, generative artificial intelligence means, robotics means, communication means, interface means, and the emotion engine, which work together to make waste sorting more efficient.

[0797] 2. System Configuration

[0798] Image recognition method: Images of garbage are captured using cameras and sensors, and the data is analyzed to identify the type of garbage.

[0799] Generative AI means: Identifies the type of garbage using acquired image data, and improves accuracy through self-learning functions.

[0800] Robotic means: Devices such as robotic arms and conveyor belts that transport identified waste to the appropriate sorting zone.

[0801] Communications: The network and communications protocols used to send and receive data to and from servers and other devices.

[0802] Interface means: An operating panel or software application that facilitates user interaction with the system.

[0803] Emotion engine: Technology that recognizes user emotions and optimizes feedback and guidance based on those emotions.

[0804] 3. Program Processing and Operation

[0805] Garbage image acquisition and recognition

[0806] 1. The device (e.g., a camera mounted on a smart trash can or a garbage truck) captures an image every time garbage is dropped in and sends the image data to a server.

[0807] 2. The server inputs the received image data into a generative AI model to identify the type of garbage.

[0808] 3. The server sends the identification result to the robotic means.

[0809] Sorting garbage

[0810] 4. Based on the identification results from the server, the terminal (robotics system) operates the robotic arm and conveyor belt to transport the waste to the appropriate sorting zone.

[0811] Home and waste incineration plant systems

[0812] 5. The terminal will provide voice guidance and display the type of garbage, and guide the user on the appropriate sorting method.

[0813] 6. Users can also use a smartphone app to take photos of trash and send them to the server, which then notifies the user of the image identification results and sorting instructions.

[0814] Emotion Recognition and Feedback

[0815] 7. The user's emotion data collected on the terminal is sent to the server via communication means.

[0816] 8. The server uses an emotion engine to recognize the user's emotional state and collect it as feedback data.

[0817] 9. The server updates the generative artificial intelligence means based on the emotion data to improve the recognition accuracy and user experience of the entire system.

[0818] 4. Specific Examples

[0819] Example 1: Smart trash can for home use

[0820] The device (a camera built into the smart trash can) captures an image of the plastic bottle.

[0821] The terminal transmits the image data to the server.

[0822] The server uses generative AI to identify it as a plastic bottle and sends instructions to the smart trash can to sort it into a plastic sorting zone.

[0823] The terminal opens the plastic lid based on the instructions and guides the user.

[0824] The device analyzes the user's emotions using an emotion engine, and if stress is felt, it changes the tone of the voice to provide guidance to reassure the user.

[0825] Example 2: Robotics system for waste incineration plants

[0826] The terminal (a camera on the conveyor belt at the waste disposal plant) captures images of the mixed waste.

[0827] The terminal transmits the image data to the server.

[0828] The server uses a generative AI to identify each type of garbage.

[0829] The server sends each identification result to the robotics system.

[0830] A robotic means (robot arm) transports each waste to the appropriate sorting zone according to the identification results.

[0831] The server uses an emotion engine to collect workers' emotional data and monitor their fatigue and stress levels.

[0832] 5. Feedback and self-learning

[0833] Users can send feedback through the smart trash can or smartphone app, and if the sorting is incorrect, the user's feedback will be reflected in the system.

[0834] The server collects feedback and emotion data and updates the generative AI model, which continuously improves its classification accuracy.

[0835] In this way, the system of the present invention not only improves the efficiency of waste sorting at homes and waste disposal plants and achieves highly accurate recycling, but also provides a comfortable experience that takes the user's emotions into consideration.

[0836] The processing flow will be explained below.

[0837] Processing flow for smart household trash cans

[0838] Step 1: Image capture of the garbage

[0839] Operation details:

[0840] The device (a camera built into the smart trash can) takes an image every time trash is dropped in.

[0841] Step 2: Sending images

[0842] Operation details:

[0843] The terminal transmits the captured image data to the server.

[0844] Step 3: Image Identification (using generative AI)

[0845] Operation details:

[0846] The server inputs the received image data into a generative AI model to identify the type of garbage.

[0847] Step 4: Notification of identification results

[0848] Operation details:

[0849] The server transmits the identification result to the terminal.

[0850] Step 5: Providing guidance on waste sorting

[0851] Operation details:

[0852] The terminal opens the lid of the appropriate sorting zone based on the type of waste identified.

[0853] The terminal guides the user on how to separate the waste by voice guidance or display on the screen.

[0854] Step 6: Collect emotion data

[0855] Operation details:

[0856] The terminal uses an emotion engine to collect the user's voice tone and movement data.

[0857] Step 7: Sending Emotion Data

[0858] Operation details:

[0859] The terminal transmits the collected emotion data to the server.

[0860] Step 8: Analyze the sentiment data

[0861] Operation details:

[0862] The server uses an emotion engine to analyze the user's emotional state and generate feedback based thereon.

[0863] Step 9: View your feedback

[0864] Operation details:

[0865] The server transmits feedback based on the analysis results to the terminal.

[0866] The terminal provides the user with a steady tone and thoughtful guidance.

[0867] Processing flow in a robotics system for waste incineration plants

[0868] Step 1: Image capture of the garbage

[0869] Operation details:

[0870] The terminal (a camera on the conveyor belt at the waste disposal plant) takes images of the waste as it flows by.

[0871] Step 2: Sending images

[0872] Operation details:

[0873] The terminal transmits the captured image data to the server.

[0874] Step 3: Image Identification (using generative AI)

[0875] Operation details:

[0876] The server inputs the received image data into a generative AI model to identify each type of garbage.

[0877] Step 4: Notification of identification results

[0878] Operation details:

[0879] The server sends each identification result to the robotics system.

[0880] Step 5: Sorting your trash

[0881] Operation details:

[0882] The terminal (robot arm) transports each piece of garbage to the appropriate sorting zone based on the identification results from the server.

[0883] Step 6: Collect worker emotion data

[0884] Operation details:

[0885] The device uses an emotion engine to analyze the worker's movements and voice and collect emotional data.

[0886] Step 7: Sending Emotion Data

[0887] Operation details:

[0888] The terminal transmits the collected emotion data to the server.

[0889] Step 8: Worker condition monitoring

[0890] Operation details:

[0891] The server uses an emotion engine to monitor workers' stress levels and fatigue and generate feedback.

[0892] Step 9: Provide monitoring results

[0893] Operation details:

[0894] The server sends the monitoring results to the terminal and notifies the workers and managers.

[0895] Processing flow for smartphone app integration

[0896] Step 1: Photograph the litter

[0897] Operation details:

[0898] Users take photos of the trash using a smartphone app.

[0899] Step 2: Send your photos

[0900] Operation details:

[0901] The terminal transmits the photographed photo data to the server.

[0902] Step 3: Image Identification (using generative AI)

[0903] Operation details:

[0904] The server inputs the received photo data into a generative AI model to identify the type of litter.

[0905] Step 4: Propose a sorting method

[0906] Operation details:

[0907] Based on the identification results, the server sends the appropriate sorting method to the terminal (smartphone app).

[0908] The terminal displays the sorting method to the user.

[0909] Step 5: Collecting sentiment data

[0910] Operation details:

[0911] The device analyzes the user's voice and actions using an emotion engine to collect emotion data.

[0912] Step 6: Sending Emotion Data

[0913] Operation details:

[0914] The terminal transmits the collected emotion data to the server.

[0915] Step 7: Analyze the sentiment data

[0916] Operation details:

[0917] The server uses an emotion engine to analyze the user's emotional state and generate feedback.

[0918] Step 8: Viewing feedback

[0919] Operation details:

[0920] The server sends optimized feedback based on the emotion data to the terminal.

[0921] The terminal displays a thoughtful guide to the user.

[0922] Feedback and self-learning process flow

[0923] Step 1: Submit your feedback

[0924] Operation details:

[0925] Users can provide feedback through the smart trash can or through a smartphone app.

[0926] Step 2: Receive and analyze feedback

[0927] Operation details:

[0928] The server analyzes the received feedback data.

[0929] Step 3: Update the AI ​​model

[0930] Operation details:

[0931] The server updates the generative AI model based on the feedback data.

[0932] Step 4: Apply the model

[0933] Operation details:

[0934] The server will use the updated AI model and apply it to future image recognition processes.

[0935] Step 5: Learning with emotion data

[0936] Operation details:

[0937] The server uses the data collected by the emotion engine to self-train the generative AI and provide feedback according to emotions.

[0938] In this way, the system of the present invention not only improves the efficiency of waste sorting at homes and waste disposal plants and achieves highly accurate recycling, but also provides a comfortable experience that takes the user's emotions into consideration.

[0939] Example 2

[0940] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0941] Conventional waste sorting systems have limitations in the accuracy and efficiency of sorting, and also have the problem of being unable to take user emotions and feedback into account. Furthermore, waste sorting systems for households and waste disposal plants have also been plagued by issues of incorrect sorting and low system learning accuracy. This has led to a demand for improved recycling rates and user experience.

[0942] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0943] In this invention, the server includes image recognition means, generative artificial intelligence means, robotics means, communication means, interface means, emotion recognition means, user feedback processing means, and self-learning means, which not only enables accurate and efficient waste sorting but also enables feedback that takes user emotions into account and continuous system learning and improvement.

[0944] "Image recognition means" refers to devices and technologies that use cameras or sensors to capture images of garbage and analyze the data to identify the type of garbage.

[0945] "Generative artificial intelligence means" refers to an artificial intelligence system that analyzes acquired image data, uses self-learning functions to identify the type of garbage, and improves identification accuracy.

[0946] "Robotics means" refers to devices such as robotic arms and conveyor belts used to transport identified waste to the appropriate sorting zone.

[0947] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[0948] "Interface means" refers to an operation panel or software application that facilitates user interaction with the system.

[0949] "Emotion recognition means" refers to technology and devices that recognize a user's emotions and optimize feedback and guidance based on those emotions.

[0950] "User feedback processing means" refers to processes and techniques for collecting feedback from users and using it to improve the system.

[0951] "Self-learning tools" refers to learning algorithms and techniques that use collected data to continuously improve the overall system's recognition accuracy and user experience.

[0952] MODE FOR CARRYING OUT THE INVENTION

[0953] System Overview

[0954] The present invention is a system that automatically separates waste in homes and waste disposal plants to improve recycling rates. In particular, the system features an emotion engine that recognizes the user's emotions, providing more appropriate feedback and guidance. The system includes image recognition means, generative artificial intelligence means, robotics means, communication means, interface means, emotion recognition means, user feedback processing means, and self-learning means, which work together to streamline waste separation.

[0955] Hardware and Software Configuration

[0956] The specific hardware and software used in the system are as follows:

[0957] 1. Devices (e.g., smart trash cans and cameras mounted on garbage trucks)

[0958] Hardware: High-resolution camera (e.g. Logitech C920)

[0959] Communication method: Wi-Fi and LTE module

[0960] Interface: Operation panel, display, microphone (e.g., Amazon Polly for voice guidance)

[0961] Emotion Recognition Technology: Microsoft's Emotion API

[0962] 2. Server

[0963] Hardware: High-performance servers (e.g. AWS EC2)

[0964] Software: Generative AI models (e.g., TensorFlow, PyTorch)

[0965] System Operation and Processing

[0966] The operation of this system is realized as follows.

[0967] The image recognition means uses a camera to capture an image of the garbage and transmits the data to a server.

[0968] The artificial intelligence generating means analyzes the image data received by the server and identifies the type of waste using a self-learning function.

[0969] The identification result is transmitted to the robotics means through the communication means.

[0970] The robotic means operates robotic arms and conveyor belts to transport identified waste to the appropriate sorting zone.

[0971] The interface means provides guidance to the user by voice or display.

[0972] The emotion recognition means recognizes the user's emotions and provides optimal guidance based on the feedback.

[0973] The user feedback processing means collects feedback from users and reflects it in improving the system.

[0974] The self-learning mechanism continuously updates the generative AI model based on collected data, improving recognition accuracy and user experience.

[0975] Specific operation example

[0976] Example of a smart trash can for home use

[0977] 1. The device (the camera built into the smart trash can) captures an image of the plastic bottle.

[0978] 2. The terminal sends the image data to the server.

[0979] 3. The server uses a generative AI model to identify it as a plastic bottle and sends instructions to the smart trash can to sort it into a plastic sorting zone.

[0980] 4. The terminal opens the plastic lid based on the instructions and guides the user.

[0981] 5. The device analyzes the user's emotions using an emotion engine, and if stress is sensed, it changes the voice tone to provide guidance to reassure the user.

[0982] Example of a robotics system for an incineration plant

[0983] 1. The terminal (a camera on the conveyor belt at the waste disposal plant) captures images of mixed waste.

[0984] 2. The terminal sends the image data to the server.

[0985] 3. The server uses a generative AI model to identify each type of garbage.

[0986] 4. The server sends each identification result to the robotics system.

[0987] 5. The robotic means (robot arm) transports each waste to the appropriate sorting zone according to the identification results.

[0988] 6. The server uses an emotion engine to collect workers' emotional data and monitor their fatigue and stress levels.

[0989] Prompt Sentence Examples

[0990] Prompt for sorting waste with a smart trash can for home:

[0991] Describe what happens to a plastic bottle after it's dropped into a smart trash can. The system includes a camera, a server, robotics tools, and an emotion engine.

[0992] Prompt for sorting garbage at an incineration plant:

[0993] Describe what happens to the mixed waste after it's placed on the conveyor belt. The system includes a camera, a server, robotics tools, and an emotion engine.

[0994] In this way, the system of the present invention not only improves the efficiency of waste sorting at homes and waste disposal plants and achieves high-precision recycling, but also provides a comfortable experience that takes user emotions into consideration.

[0995] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0996] Step 1:

[0997] The device (the built-in camera of the smart trash can) automatically captures images when trash is dropped in. The input is a physical image of the trash, which is captured by the camera and converted into image data in JPEG format. The camera uses a high-resolution camera (e.g., Logitech C920) to adjust the exposure and focus in real time.

[0998] Step 2:

[0999] The device sends the acquired image data to the server via the Wi-Fi module. The input is the JPEG image data acquired in step 1, which is encrypted and sent. The data is sent securely using TLS (Transport Layer Security).

[1000] Step 3:

[1001] The server inputs the received image data into the generative AI model. The input is JPEG image data, which is input to a virtual machine running TensorFlow on the server. The generative AI model analyzes the image data and performs data calculations to identify the type of garbage. The output is the result of identifying the type of garbage.

[1002] Step 4:

[1003] The server converts the identification results into JSON format and sends them back to the device via a communication method. The input is the identification results from the TensorFlow model, which are serialized into JSON format. The output is the JSON-formatted identification results, which are sent via the secure WebSocket protocol.

[1004] Step 5:

[1005] The terminal (robotics system) operates the robotic arm and conveyor belt based on the identification results. The input is the identification results in JSON format from the server, which is analyzed to generate specific operation instructions. For example, it operates an ABB IRB 6700 robotic arm to perform a specific operation, such as transporting aluminum cans to a metal sorting zone. The output is that the waste is transported to the appropriate sorting zone.

[1006] Step 6:

[1007] The device provides the user with the sorting results and instructions through voice guidance and a display. The inputs are the JSON formatted identification results and text for the voice guidance, which are converted into speech using speech synthesis technology (e.g., Amazon Polly). The output is a message saying, "This garbage is a plastic bottle. Please use a plastic recycling lid."

[1008] Step 7:

[1009] The device collects the user's emotional data and sends it to the server. The input includes facial expressions and voices while the user is performing the classification activity, and data is collected using an facial recognition camera and microphone. This data is analyzed using Microsoft's Emotion API and converted into emotional data. The output is the analyzed emotional data, which is sent to the server via secure communication.

[1010] Step 8:

[1011] The server updates the generative AI model based on emotion data and user feedback. The inputs are emotion data obtained from the Emotion API and user feedback data, which are analyzed to adjust the parameters of the generative AI model. The data is then processed to retrain the model's neural network and improve its recognition accuracy. The output is an updated generative AI model, which improves recognition accuracy and the user experience from the next time onwards.

[1012] (Application example 2)

[1013] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1014] Traditionally, parts sorting and quality inspection in factories have been largely manual and inefficient. Furthermore, workers' stress and emotional states have not been properly managed, leading to a deterioration in the working environment. This has led to reduced production efficiency and increased likelihood of work errors, affecting overall work efficiency. To solve these problems, a system is needed that can automatically identify and sort parts while simultaneously managing the emotional states of workers.

[1015] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1016] In this invention, the server includes image recognition means, generative artificial intelligence means, robotics means, communication means, interface means, emotion recognition means, and emotion-based feedback means, thereby improving the efficiency of automatic part identification and sorting, and enabling real-time monitoring of the emotional state of workers and appropriate feedback.

[1017] An "image recognition means" is a device that uses a camera or sensor to capture an image of an object, analyzes the data, and identifies the type and condition of the object.

[1018] The "generative artificial intelligence means" is an artificial intelligence system that identifies the target object based on the acquired image data and improves the accuracy of identification through its self-learning function.

[1019] "Robotics means" refers to devices such as robotic arms and conveyor belts that transport identified objects to the appropriate location.

[1020] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[1021] "Interface means" refers to an operation panel or software application that facilitates user interaction with the system.

[1022] "Emotion recognition means" refers to a device or software that analyzes a user's facial image to determine the user's emotional state.

[1023] The "means for providing feedback based on emotions" refers to a device or software that provides appropriate guidance or feedback to the user based on the results of emotion recognition.

[1024] This invention is a system for improving the efficiency of parts sorting and quality inspection in factories. It also monitors the emotional state of workers in real time and manages their stress. The specific system configuration, usage, and operational procedures are described below.

[1025] 1. System Configuration

[1026] The system includes the following main elements:

[1027] Image recognition means: The camera in the smart glasses captures an image of the part.

[1028] Generative AI means: Analyzes acquired image data and identifies the type and quality of parts. For example, it has the ability to determine whether a product is good or defective.

[1029] Robotic means: Robotic arms and conveyor belts to transport identified parts to the appropriate location.

[1030] Communications: The network and communications protocols used to send and receive data to and from servers and other devices.

[1031] Interface means: An operation panel or software application that allows the worker to interact with the system. This can be the display or voice guidance on the smart glasses.

[1032] Emotion recognition means: Software that captures facial images of workers using the camera in the smart glasses and analyzes their emotional state.

[1033] Means of providing emotion-based feedback: Analyze the emotional state and provide feedback based on stress, fatigue, etc. via audio guidance, etc.

[1034] 2. Program Processing

[1035] Image Acquisition and Recognition

[1036] The device (the camera in the smart glasses) captures an image of the part. Image recognition means is used to analyze the image data and identify the type and condition (good or bad) of the part.

[1037] The server sends the image data to a generative AI model for part identification, which automatically identifies the part type and quality.

[1038] Emotion Recognition and Feedback

[1039] The device (the camera in the smart glasses) captures facial images of the worker, which are then analyzed by emotion recognition means to determine the worker's emotional state (especially their stress level).

[1040] Based on the results of emotion recognition, the server analyzes the data using generative artificial intelligence and provides appropriate guidance to the worker through emotion-based feedback, such as "take a break" if the worker is feeling high stress.

[1041] As a concrete example, imagine a scene where a worker is wearing smart glasses and working on a factory production line. Every time a part comes through, the camera in the smart glasses takes an image of the part, and the AI ​​immediately identifies whether it is a good or defective part. If a defective part is detected, a voice message will be played saying, "A defective part has been detected. Please remove it." The system will also analyze the worker's facial expression, and if the worker's stress level is high, a voice message will be played saying, "You are under a lot of stress. Please take a short break."

[1042] 3. Example prompts

[1043] Input image: An image of a part on a factory production line

[1044] Task: Identify whether the part is good or bad.

[1045] Input image: Face image of a worker

[1046] Task: Identify emotions and stress levels from workers' facial expressions.

[1047] This system will significantly improve the efficiency of automatic identification and sorting of parts in factories, and by monitoring the emotional state of workers and providing appropriate feedback, it will improve the working environment and increase production efficiency.

[1048] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1049] Step 1:

[1050] The device uses the camera in the smart glasses to capture images of parts on the factory production line. The input is image data of the parts, and the output is an image file.

[1051] Step 2:

[1052] The terminal transmits the acquired image data to the server using a communication means. The input is the image file, and the output is the result of the data transmission to the server.

[1053] Step 3:

[1054] The server inputs the received image data into the artificial intelligence generating means to identify the type and quality of the part. The input is the image file, and the output is the label of the part's identification result (e.g., good, defective).

[1055] Step 4:

[1056] The server sends the identification results to the robotics means and issues instructions for automatic sorting. The input is the label of the part identification result, and the output is operation instructions for the robot arm and conveyor belt.

[1057] Step 5:

[1058] The terminal acquires the facial image of the worker using the camera in the smart glasses. The input is the facial image data, and the output is a facial image file.

[1059] Step 6:

[1060] The terminal transmits the facial image data to the server using a communication means. The input is a facial image file, and the output is the result of the data transmission to the server.

[1061] Step 7:

[1062] The server inputs the received facial image data into the emotion recognition means to determine the worker's emotional state and stress level. The input is a facial image file, and the output is a label of the emotional state and a numerical value of the stress level.

[1063] Step 8:

[1064] The server generates a feedback message based on the emotional state and stress level using a generation artificial intelligence means and transmits it to the terminal, where the input is the label of the emotional state and the numerical value of the stress level, and the output is the feedback message.

[1065] Step 9:

[1066] The terminal provides the worker with a feedback message through an interface means such as voice guidance or display, etc. The input is the feedback message, and the output is the voice or display.

[1067] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1068] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1069] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1070] [Third embodiment]

[1071] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1072] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1073] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1074] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1075] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1076] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1077] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1078] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1079] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1080] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1081] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1082] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1083] 1. System Overview

[1084] The present invention is a system for automatically separating waste in homes and waste disposal plants, thereby improving recycling rates. The system includes image recognition means, generative artificial intelligence means, robotics means, communication means, and interface means, which work together to make waste separation more efficient.

[1085] 2. System Configuration

[1086] Image recognition method: Images of garbage are captured using cameras and sensors, and the data is analyzed to identify the type of garbage.

[1087] Generative AI means: Identifies the type of garbage using acquired image data, and improves accuracy through self-learning functions.

[1088] Robotic means: Devices such as robotic arms and conveyor belts that transport identified waste to the appropriate sorting zone.

[1089] Communications: The network and communications protocols used to send and receive data to and from servers and other devices.

[1090] Interface means: An operating panel or software application that facilitates user interaction with the system.

[1091] 3. Program Processing and Operation

[1092] Garbage image acquisition and recognition

[1093] 1. The device (e.g., a camera mounted on a smart trash can or a garbage truck) captures an image every time garbage is dropped in and sends the image data to a server.

[1094] 2. The server inputs the received image data into a generative AI model to identify the type of garbage.

[1095] 3. The server sends the identification result to the robotic means.

[1096] Sorting garbage

[1097] 4. Based on the identification results from the server, the terminal (robotics system) operates the robotic arm and conveyor belt to transport the waste to the appropriate sorting zone.

[1098] Home and waste incineration plant systems

[1099] 5. The terminal will provide voice guidance and display the type of garbage, and guide the user on the appropriate sorting method.

[1100] 6. Users can also use a smartphone app to take photos of trash and send them to the server, which then notifies the user of the image identification results and sorting instructions.

[1101] 4. Specific Examples

[1102] Example 1: Smart trash can for home use

[1103] The device (a camera built into the smart trash can) captures an image of the plastic bottle.

[1104] The terminal transmits the image data to the server.

[1105] The server uses generative AI to identify it as a plastic bottle and sends instructions to the smart trash can to sort it into a plastic sorting zone.

[1106] The terminal opens the plastic lid based on the instructions and guides the user.

[1107] Example 2: Robotics system for waste incineration plants

[1108] The terminal (a camera on the conveyor belt at the waste disposal plant) captures images of the mixed waste.

[1109] The terminal transmits the image data to the server.

[1110] The server uses a generative AI to identify each type of garbage.

[1111] The server sends each identification result to the robotics system.

[1112] A robotic means (robot arm) transports each waste to the appropriate sorting zone according to the identification results.

[1113] 5. Feedback and self-study

[1114] Users can send feedback through the smart trash can or smartphone app, and if the sorting is incorrect, the user's feedback will be reflected in the system.

[1115] The server collects feedback data and updates the generative AI model, which continuously improves its classification accuracy.

[1116] In this way, the system of the present invention improves the efficiency of waste sorting at homes and waste disposal plants, and realizes highly accurate recycling.

[1117] The processing flow will be explained below.

[1118] Processing flow for smart household trash cans

[1119] Step 1: Image capture of the garbage

[1120] Operation details:

[1121] The device (a camera built into the smart trash can) takes an image every time trash is dropped in.

[1122] Step 2: Sending images

[1123] Operation details:

[1124] The terminal transmits the captured image data to the server.

[1125] Step 3: Image Identification (using generative AI)

[1126] Operation details:

[1127] The server inputs the received image data into a generative AI model to identify the type of garbage.

[1128] Step 4: Notification of identification results

[1129] Operation details:

[1130] The server transmits the identification result to the terminal.

[1131] Step 5: Providing guidance on waste sorting

[1132] Operation details:

[1133] The terminal opens the lid of the appropriate sorting zone based on the type of waste identified.

[1134] The terminal guides the user on how to separate the waste by voice guidance or display on the screen.

[1135] Processing flow in a robotics system for waste incineration plants

[1136] Step 1: Image capture of the garbage

[1137] Operation details:

[1138] The terminal (a camera on the conveyor belt at the waste disposal plant) takes images of the waste as it flows by.

[1139] Step 2: Sending images

[1140] Operation details:

[1141] The terminal transmits the captured image data to the server.

[1142] Step 3: Image Identification (using generative AI)

[1143] Operation details:

[1144] The server inputs the received image data into a generative AI model to identify each type of garbage.

[1145] Step 4: Notification of identification results

[1146] Operation details:

[1147] The server sends each identification result to the robotics system.

[1148] Step 5: Sorting your trash

[1149] Operation details:

[1150] The terminal (robot arm) transports each piece of garbage to the appropriate sorting zone based on the identification results from the server.

[1151] Processing flow for smartphone app integration

[1152] Step 1: Photograph the litter

[1153] Operation details:

[1154] Users take photos of the trash using a smartphone app.

[1155] Step 2: Send your photos

[1156] Operation details:

[1157] The terminal transmits the photographed photo data to the server.

[1158] Step 3: Image Identification (using generative AI)

[1159] Operation details:

[1160] The server inputs the received image data into a generative AI model to identify the type of garbage.

[1161] Step 4: Propose a sorting method

[1162] Operation details:

[1163] Based on the identification results, the server sends the appropriate sorting method to the terminal (smartphone app).

[1164] The terminal displays the sorting method to the user.

[1165] Feedback and self-learning process flow

[1166] Step 1: Submit your feedback

[1167] Operation details:

[1168] Users can provide feedback on their sorting results via a smartphone app or smart trash can.

[1169] Step 2: Receive and analyze feedback

[1170] Operation details:

[1171] The server analyzes the received feedback data.

[1172] Step 3: Update the AI ​​model

[1173] Operation details:

[1174] The server updates the generative AI model based on the feedback data.

[1175] Step 4: Apply the model

[1176] Operation details:

[1177] The server will use the updated AI model and apply it to future image recognition processes.

[1178] Through these processing steps, the sorting accuracy of the entire system will improve, aiming to increase recycling rates and realize a sustainable society.

[1179] Example 1

[1180] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1181] Conventional waste sorting at homes and waste disposal plants relies on manual labor or simple machines, resulting in low sorting accuracy and limiting the improvement of recycling rates. Manual sorting is also labor-intensive and time-consuming, and has many issues with efficiency. Furthermore, when incorrect sorting occurs, it is difficult to incorporate user feedback into the system, and the system lacks a self-learning function. There is a need for a system that can solve these problems and achieve high-precision, efficient waste sorting.

[1182] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1183] In this invention, the server includes a means for transmitting image data from the terminal to the server, a means for analyzing image data in the server, and a means for transmitting identification results from the server to the robotics means. This enables highly accurate identification of waste types. The server also includes a means for receiving feedback data from users and a means for self-learning and updating the generative artificial intelligence model, allowing for continuous improvement in identification accuracy through feedback. Furthermore, the robotics means automatically transports waste to the sorting zone, significantly improving work efficiency. This system can be used for both home and waste disposal plants, contributing to increased recycling rates.

[1184] "Image recognition means" refers to a device or software that analyzes acquired image data and identifies the type of garbage.

[1185] "Generative artificial intelligence means" refers to an AI model and related system that uses acquired image data to identify the type of garbage and improves identification accuracy through self-learning functions.

[1186] "Robotics means" refers to devices such as robotic arms and conveyor belts that transport identified waste to the appropriate sorting zone.

[1187] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[1188] "Interface means" refers to an operation panel or software application that facilitates user interaction with the system.

[1189] The "means for transmitting image data from the terminal to the server" is a function for transmitting image data captured by the terminal to the server in real time.

[1190] "Image data analysis means on the server" is a function that inputs the image data received by the server into a generative AI model, analyzes it, and identifies the type of garbage.

[1191] The "means for transmitting identification results from the server to the robotics means" is a function for transmitting the identification results analyzed by the server to the robotics means.

[1192] The "means for receiving feedback data from users" is a function that receives feedback data provided by users via smartphone apps or the like on the server.

[1193] The "self-learning update means for the generative AI model" is a function that updates the learning of the generative AI model using received feedback data to improve the recognition accuracy.

[1194] System Overview

[1195] The present invention is a system for automatically separating waste at homes and waste disposal plants to improve recycling rates. This system operates by linking together image recognition means, generative artificial intelligence means, robotics means, communication means, and interface means.

[1196] Hardware and software used

[1197] Image recognition method: Images of garbage are captured using cameras or sensors, and the data is analyzed. Specifically, a high-resolution camera (e.g., Sony's IMX577 sensor) is used.

[1198] Generative AI method: The acquired image data is input into a generative AI model (e.g., a deep learning model using TensorFlow or PyTorch) to identify the type of garbage.

[1199] Robotic means: using robotic arms or conveyor belts that transport identified waste to the appropriate sorting zone. Examples include robotic arms from KUKA and Fanuc.

[1200] Communication method: Uses a network (e.g., Wi-Fi module and communication protocols such as HTTP, WebSocket, and MQTT) to send and receive data to and from servers and other devices.

[1201] Interface means: An operation panel or software application that makes it easy for the user to interact with the system. Examples include Android and iOS applications.

[1202] Image acquisition and identification of debris

[1203] 1. The device uses cameras installed in smart trash bins or on conveyor belts at waste disposal plants to capture images every time trash is dropped in.

[1204] 2. The device transmits the acquired image data to the server in real time.

[1205] 3. The server inputs the received image data into a generative AI model to identify the type of garbage. Because the generative AI model has been pre-trained on a large dataset of garbage images, it is able to identify the type of garbage with high accuracy.

[1206] 4. The server sends the identification result to the robotic means.

[1207] Sorting garbage

[1208] 5. The terminal (robotics system) receives the identification results from the server and begins to act based on that information. Specifically, the robotic arm moves and transports the identified waste to the appropriate sorting zone.

[1209] Display of sorting instructions

[1210] 6. The terminal provides the user with guidance on proper waste separation, using a display and a voice guidance system.

[1211] User Feedback

[1212] 7. Users send feedback to the system through a smartphone app. For example, if a garbage item is misclassified, they can send feedback from the app, such as, "This garbage is not a plastic bottle, but an aluminum can."

[1213] Learning from feedback data

[1214] 8. The server aggregates user feedback data and retrains the generative AI model to continuously improve its classification accuracy, periodically updating the model using online learning techniques.

[1215] Specific examples

[1216] Example 1: Smart trash can for home use

[1217] The device (a camera built into the smart trash can) captures an image of the plastic bottle.

[1218] The terminal transmits this image data to the server.

[1219] The server uses a generative AI model to identify the bottle as a PET bottle and sort it into a separate zone for plastics.

[1220] The terminal opens the plastic lid and guides the user.

[1221] Example 2: Robotics system for waste incineration plants

[1222] The terminal (a camera on the conveyor belt of the waste disposal plant) captures images of the mixed waste.

[1223] The terminal transmits the image data to the server.

[1224] The server uses a generative AI model to identify each type of garbage.

[1225] The server sends each identification result to the robotics system.

[1226] A robotic means (robot arm) transports each waste to the appropriate sorting zone according to the identification results.

[1227] Prompt Sentence Examples

[1228] "What is this garbage?"

[1229] "Please identify the trash in this image."

[1230] "Which sorting zone should this garbage be sorted into?"

[1231] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1232] Step 1:

[1233] The device uses a camera to capture images of the garbage. The device uses a high-resolution camera to capture pixel information. The captured image data is temporarily stored in the camera's memory.

[1234] Input: Physical presence of litter

[1235] Data processing: Image capture using a high-resolution camera

[1236] Output: High-resolution image data

[1237] Step 2:

[1238] The image data acquired by the device is sent to the server in real time using Wi-Fi or other communication protocols (e.g., HTTP or WebSocket).

[1239] Input: High-resolution image data

[1240] Data processing: Converting image data into digital format and sending it

[1241] Output: Image data sent to the server

[1242] Step 3:

[1243] The server inputs the received image data into a generative AI model for data analysis. The generative AI model uses deep learning algorithms such as TensorFlow and PyTorch. The model analyzes the image data and identifies the type of garbage.

[1244] Input: Image data

[1245] Data processing / calculation: Image analysis and garbage type identification using generative AI models

[1246] Output: Classification result data (e.g. "PET bottle")

[1247] Step 4:

[1248] The server transmits the identification results to the robotic means using a lightweight communication protocol such as MQTT, which allows for low-latency communication.

[1249] Input: Classification result data

[1250] Data processing: Convert the identification result data into digital format and send it

[1251] Output: Identification result data sent to robotic means

[1252] Step 5:

[1253] The terminal (robotics system) receives the identification results from the server and starts the waste sorting process based on them. Specifically, the robotic arm moves and transports the identified waste to the appropriate sorting zone. The robotic arm used could be a product from KUKA or Fanuc, for example.

[1254] Input: Classification result data

[1255] Data processing: Controlling robot behavior based on classification results

[1256] Output: Execution of waste sorting work (transfer of waste)

[1257] Step 6:

[1258] The terminal provides users with guidance on how to separate their waste. It uses a display and a voice guidance system to provide specific instructions on how to separate waste. The terminal's display uses a highly visible LCD panel, and voice information is provided through a speaker.

[1259] Input: Classification result data

[1260] Data processing: Creating content for waste sorting guidance

[1261] Output: Display or audio guidance for sorting

[1262] Step 7:

[1263] Users can provide feedback through a smartphone app. For example, if a garbage item is misclassified, they can provide feedback such as, "This garbage is not a plastic bottle, but an aluminum can." The app is developed using React Native and Flutter.

[1264] Input: User feedback information

[1265] Data processing: digitization and transmission of feedback information

[1266] Output: Feedback data sent to the server

[1267] Step 8:

[1268] The server collects the received feedback data and uses it to retrain the generative AI model. The feedback data is added to the model training dataset to improve classification accuracy. The model is periodically updated using online learning techniques.

[1269] Input: User feedback data

[1270] Data processing / calculation: Aggregating and generating feedback information Retraining AI models

[1271] Output: An updated generative AI model

[1272] Through these steps, the system automatically separates waste with high accuracy and efficiency, improving recycling rates.

[1273] (Application example 1)

[1274] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1275] Conventional waste sorting systems are primarily used in households and waste disposal plants, making it difficult to efficiently separate waste generated within the plant. Issues also existed, such as problems with identification accuracy and reduced recycling efficiency due to incorrect sorting. Furthermore, the system's self-learning capabilities were insufficient to improve accuracy, and effective use of user feedback was also required.

[1276] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1277] In this invention, the server includes an image acquisition means, a recognition means using a generative AI model, a robot control means, a communication means, and an interface means. This enables automatic and highly accurate waste sorting within the factory. Specifically, efficient waste sorting is achieved by coordinating image acquisition from a camera device, waste type identification using a generative AI model, and waste transportation by a robot arm. In addition, a self-learning function continuously improves the accuracy of identification, and by incorporating user feedback as data, even more accurate recycling is possible.

[1278] "Image acquisition means" refers to a device or sensor used to acquire image data from an object.

[1279] "Recognition means using generative AI models" refers to a method of using artificial intelligence to analyze acquired image data and identify the type of object.

[1280] "Robot control means" refers to a system or device that controls an automatically operating robot to perform a specific action or task.

[1281] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[1282] "Interface means" refers to devices and programs such as operation panels and software applications that make it easier for users to operate the system.

[1283] "Industrial waste sorting equipment" means equipment for automatically identifying waste generated within a factory and transporting it to the appropriate sorting zone.

[1284] "Artificial intelligence with self-learning capabilities" refers to artificial intelligence that has the ability to continuously improve its recognition accuracy based on acquired data and user feedback.

[1285] This invention relates to a smart factory waste sorting robot system that automatically sorts waste generated in factories and improves recycling efficiency. The system includes an image acquisition means, a recognition means using a generative AI model, a robot control means, a communication means, an interface means, an industrial waste sorting device, and an artificial intelligence with a self-learning function, which work together to improve the efficiency of waste sorting.

[1286] The image acquisition means provided by the server acquires image data of waste using a camera device. Examples of hardware used include a camera device such as the Logitech C920. This camera device is installed on the conveyor belt where waste is mixed in the factory and takes images periodically.

[1287] The server inputs the acquired image data into a recognition means using a generative AI model to identify the type of waste. This generative AI model is built using TensorFlow Keras and performs highly accurate waste identification. The server then sends the identification results to the robot control means, which provides instructions for transporting the waste to the appropriate sorting zone.

[1288] The robot control means provided by the terminal operates the robot arm based on instructions from the server and transports the waste to the appropriate sorting zone. An example of the robot arm used is the UR5 robot arm. The robot control means is also equipped with an industrial waste sorting device, enabling efficient transport of waste.

[1289] The communication means includes the network and communication protocols for transmitting and receiving data between the server, terminal, and other devices, which allows the entire system to work together.

[1290] The interface means includes an operation panel and software applications that allow users to easily operate the system, allowing them to provide feedback to the system and contribute to improving its accuracy. Furthermore, the system has a self-learning function that allows it to continuously improve its classification accuracy based on user feedback.

[1291] As a concrete example, imagine a conveyor belt with mixed waste. A camera on this conveyor belt periodically takes images and sends them to a generative AI model for identification. For example, use the following prompt:

[1292] Input image: (uploaded garbage image)

[1293] ---

[1294] Question: What type of trash is in this image?

[1295] ---

[1296] Example output:

[1297] plastic

[1298] metal

[1299] General waste

[1300] As described above, the present invention enables waste materials in a factory to be separated efficiently and with high accuracy, thereby improving recycling efficiency.

[1301] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1302] Step 1:

[1303] The terminal uses a camera device to capture images of the waste on the conveyor belt. The input is image data from the camera device and the output is the captured image. This image data is sent to the next processing step.

[1304] Step 2:

[1305] The device sends the acquired image data to the server. The input is image data from the camera device, and the output is the image data sent to the server. The server receives this data and analyzes it using a generative AI model.

[1306] Step 3:

[1307] The server inputs the image data into a generative AI model to identify the type of waste. The input is the submitted image data, and the output is the identified waste type (e.g., plastic, metal, general waste). The identification result is sent to the next processing step.

[1308] Step 4:

[1309] The server transmits the identification result to the robot control means, where the input is the identification result regarding the type of waste, and the output is instruction data for the robot control means, including instructions for transporting the waste to the appropriate sorting zone.

[1310] Step 5:

[1311] The terminal controls the robot arm to transport the identified waste to the appropriate sorting zone. The input is instruction data from the server, and the output is the waste transported to the appropriate sorting zone. The robot arm operates based on the identification result.

[1312] Step 6:

[1313] The user operates the system using an interface means and provides feedback as needed. The input is the user's operation and feedback data, and the output is the feedback information sent to the system. This feedback information is collected by the server and used to update the self-learning artificial intelligence.

[1314] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1315] 1. System Overview

[1316] The present invention is a system for automatically sorting waste in homes and waste disposal plants to improve recycling rates. In particular, this system features an emotion engine that recognizes the user's emotions, providing more appropriate feedback and guidance. The system includes image recognition means, generative artificial intelligence means, robotics means, communication means, interface means, and the emotion engine, which work together to make waste sorting more efficient.

[1317] 2. System Configuration

[1318] Image recognition method: Images of garbage are captured using cameras and sensors, and the data is analyzed to identify the type of garbage.

[1319] Generative AI means: Identifies the type of garbage using acquired image data, and improves accuracy through self-learning functions.

[1320] Robotic means: Devices such as robotic arms and conveyor belts that transport identified waste to the appropriate sorting zone.

[1321] Communications: The network and communications protocols used to send and receive data to and from servers and other devices.

[1322] Interface means: An operating panel or software application that facilitates user interaction with the system.

[1323] Emotion engine: Technology that recognizes user emotions and optimizes feedback and guidance based on those emotions.

[1324] 3. Program Processing and Operation

[1325] Garbage image acquisition and recognition

[1326] 1. The device (e.g., a camera mounted on a smart trash can or a garbage truck) captures an image every time garbage is dropped in and sends the image data to a server.

[1327] 2. The server inputs the received image data into a generative AI model to identify the type of garbage.

[1328] 3. The server sends the identification result to the robotic means.

[1329] Sorting garbage

[1330] 4. Based on the identification results from the server, the terminal (robotics system) operates the robotic arm and conveyor belt to transport the waste to the appropriate sorting zone.

[1331] Home and waste incineration plant systems

[1332] 5. The terminal will provide voice guidance and display the type of garbage, and guide the user on the appropriate sorting method.

[1333] 6. Users can also use a smartphone app to take photos of trash and send them to the server, which then notifies the user of the image identification results and sorting instructions.

[1334] Emotion Recognition and Feedback

[1335] 7. The user's emotion data collected on the terminal is sent to the server via communication means.

[1336] 8. The server uses an emotion engine to recognize the user's emotional state and collect it as feedback data.

[1337] 9. The server updates the generative artificial intelligence means based on the emotion data to improve the recognition accuracy and user experience of the entire system.

[1338] 4. Specific Examples

[1339] Example 1: Smart trash can for home use

[1340] The device (a camera built into the smart trash can) captures an image of the plastic bottle.

[1341] The terminal transmits the image data to the server.

[1342] The server uses generative AI to identify it as a plastic bottle and sends instructions to the smart trash can to sort it into a plastic sorting zone.

[1343] The terminal opens the plastic lid based on the instructions and guides the user.

[1344] The device analyzes the user's emotions using an emotion engine, and if stress is felt, it changes the tone of the voice to provide guidance to reassure the user.

[1345] Example 2: Robotics system for waste incineration plants

[1346] The terminal (a camera on the conveyor belt at the waste disposal plant) captures images of the mixed waste.

[1347] The terminal transmits the image data to the server.

[1348] The server uses a generative AI to identify each type of garbage.

[1349] The server sends each identification result to the robotics system.

[1350] A robotic means (robot arm) transports each waste to the appropriate sorting zone according to the identification results.

[1351] The server uses an emotion engine to collect workers' emotional data and monitor their fatigue and stress levels.

[1352] 5. Feedback and self-learning

[1353] Users can send feedback through the smart trash can or smartphone app, and if the sorting is incorrect, the user's feedback will be reflected in the system.

[1354] The server collects feedback and emotion data and updates the generative AI model, which continuously improves its classification accuracy.

[1355] In this way, the system of the present invention not only improves the efficiency of waste sorting at homes and waste disposal plants and achieves highly accurate recycling, but also provides a comfortable experience that takes the user's emotions into consideration.

[1356] The processing flow will be explained below.

[1357] Processing flow for smart household trash cans

[1358] Step 1: Image capture of the garbage

[1359] Operation details:

[1360] The device (a camera built into the smart trash can) takes an image every time trash is dropped in.

[1361] Step 2: Sending images

[1362] Operation details:

[1363] The terminal transmits the captured image data to the server.

[1364] Step 3: Image Identification (using generative AI)

[1365] Operation details:

[1366] The server inputs the received image data into a generative AI model to identify the type of garbage.

[1367] Step 4: Notification of identification results

[1368] Operation details:

[1369] The server transmits the identification result to the terminal.

[1370] Step 5: Providing guidance on waste sorting

[1371] Operation details:

[1372] The terminal opens the lid of the appropriate sorting zone based on the type of waste identified.

[1373] The terminal guides the user on how to separate the waste by voice guidance or display on the screen.

[1374] Step 6: Collect emotion data

[1375] Operation details:

[1376] The terminal uses an emotion engine to collect the user's voice tone and movement data.

[1377] Step 7: Sending Emotion Data

[1378] Operation details:

[1379] The terminal transmits the collected emotion data to the server.

[1380] Step 8: Analyze the sentiment data

[1381] Operation details:

[1382] The server uses an emotion engine to analyze the user's emotional state and generate feedback based thereon.

[1383] Step 9: View your feedback

[1384] Operation details:

[1385] The server transmits feedback based on the analysis results to the terminal.

[1386] The terminal provides the user with a steady tone and thoughtful guidance.

[1387] Processing flow in a robotics system for waste incineration plants

[1388] Step 1: Image capture of the garbage

[1389] Operation details:

[1390] The terminal (a camera on the conveyor belt at the waste disposal plant) takes images of the waste as it flows by.

[1391] Step 2: Sending images

[1392] Operation details:

[1393] The terminal transmits the captured image data to the server.

[1394] Step 3: Image Identification (using generative AI)

[1395] Operation details:

[1396] The server inputs the received image data into a generative AI model to identify each type of garbage.

[1397] Step 4: Notification of identification results

[1398] Operation details:

[1399] The server sends each identification result to the robotics system.

[1400] Step 5: Sorting your trash

[1401] Operation details:

[1402] The terminal (robot arm) transports each piece of garbage to the appropriate sorting zone based on the identification results from the server.

[1403] Step 6: Collect worker emotion data

[1404] Operation details:

[1405] The device uses an emotion engine to analyze the worker's movements and voice and collect emotional data.

[1406] Step 7: Sending Emotion Data

[1407] Operation details:

[1408] The terminal transmits the collected emotion data to the server.

[1409] Step 8: Worker condition monitoring

[1410] Operation details:

[1411] The server uses an emotion engine to monitor workers' stress levels and fatigue and generate feedback.

[1412] Step 9: Provide monitoring results

[1413] Operation details:

[1414] The server sends the monitoring results to the terminal and notifies the workers and managers.

[1415] Processing flow for smartphone app integration

[1416] Step 1: Photograph the litter

[1417] Operation details:

[1418] Users take photos of the trash using a smartphone app.

[1419] Step 2: Send your photos

[1420] Operation details:

[1421] The terminal transmits the photographed photo data to the server.

[1422] Step 3: Image Identification (using generative AI)

[1423] Operation details:

[1424] The server inputs the received photo data into a generative AI model to identify the type of litter.

[1425] Step 4: Propose a sorting method

[1426] Operation details:

[1427] Based on the identification results, the server sends the appropriate sorting method to the terminal (smartphone app).

[1428] The terminal displays the sorting method to the user.

[1429] Step 5: Collecting sentiment data

[1430] Operation details:

[1431] The device analyzes the user's voice and actions using an emotion engine to collect emotion data.

[1432] Step 6: Sending Emotion Data

[1433] Operation details:

[1434] The terminal transmits the collected emotion data to the server.

[1435] Step 7: Analyze the sentiment data

[1436] Operation details:

[1437] The server uses an emotion engine to analyze the user's emotional state and generate feedback.

[1438] Step 8: Viewing feedback

[1439] Operation details:

[1440] The server sends optimized feedback based on the emotion data to the terminal.

[1441] The terminal displays a thoughtful guide to the user.

[1442] Feedback and self-learning process flow

[1443] Step 1: Submit your feedback

[1444] Operation details:

[1445] Users can provide feedback through the smart trash can or through a smartphone app.

[1446] Step 2: Receive and analyze feedback

[1447] Operation details:

[1448] The server analyzes the received feedback data.

[1449] Step 3: Update the AI ​​model

[1450] Operation details:

[1451] The server updates the generative AI model based on the feedback data.

[1452] Step 4: Apply the model

[1453] Operation details:

[1454] The server will use the updated AI model and apply it to future image recognition processes.

[1455] Step 5: Learning with emotion data

[1456] Operation details:

[1457] The server uses the data collected by the emotion engine to self-train the generative AI and provide feedback according to emotions.

[1458] In this way, the system of the present invention not only improves the efficiency of waste sorting at homes and waste disposal plants and achieves highly accurate recycling, but also provides a comfortable experience that takes the user's emotions into consideration.

[1459] Example 2

[1460] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1461] Conventional waste sorting systems have limitations in the accuracy and efficiency of sorting, and also have the problem of being unable to take user emotions and feedback into account. Furthermore, waste sorting systems for households and waste disposal plants have also been plagued by issues of incorrect sorting and low system learning accuracy. This has led to a demand for improved recycling rates and user experience.

[1462] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1463] In this invention, the server includes image recognition means, generative artificial intelligence means, robotics means, communication means, interface means, emotion recognition means, user feedback processing means, and self-learning means, which not only enables accurate and efficient waste sorting but also enables feedback that takes user emotions into account and continuous system learning and improvement.

[1464] "Image recognition means" refers to devices and technologies that use cameras or sensors to capture images of garbage and analyze the data to identify the type of garbage.

[1465] "Generative artificial intelligence means" refers to an artificial intelligence system that analyzes acquired image data, uses self-learning functions to identify the type of garbage, and improves identification accuracy.

[1466] "Robotics means" refers to devices such as robotic arms and conveyor belts used to transport identified waste to the appropriate sorting zone.

[1467] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[1468] "Interface means" refers to an operation panel or software application that facilitates user interaction with the system.

[1469] "Emotion recognition means" refers to technology and devices that recognize a user's emotions and optimize feedback and guidance based on those emotions.

[1470] "User feedback processing means" refers to processes and techniques for collecting feedback from users and using it to improve the system.

[1471] "Self-learning tools" refers to learning algorithms and techniques that use collected data to continuously improve the overall system's recognition accuracy and user experience.

[1472] MODE FOR CARRYING OUT THE INVENTION

[1473] System Overview

[1474] The present invention is a system that automatically separates waste in homes and waste disposal plants to improve recycling rates. In particular, the system features an emotion engine that recognizes the user's emotions, providing more appropriate feedback and guidance. The system includes image recognition means, generative artificial intelligence means, robotics means, communication means, interface means, emotion recognition means, user feedback processing means, and self-learning means, which work together to streamline waste separation.

[1475] Hardware and Software Configuration

[1476] The specific hardware and software used in the system are as follows:

[1477] 1. Devices (e.g., smart trash cans and cameras mounted on garbage trucks)

[1478] Hardware: High-resolution camera (e.g. Logitech C920)

[1479] Communication method: Wi-Fi and LTE module

[1480] Interface: Operation panel, display, microphone (e.g., Amazon Polly for voice guidance)

[1481] Emotion Recognition Technology: Microsoft's Emotion API

[1482] 2. Server

[1483] Hardware: High-performance servers (e.g. AWS EC2)

[1484] Software: Generative AI models (e.g., TensorFlow, PyTorch)

[1485] System Operation and Processing

[1486] The operation of this system is realized as follows.

[1487] The image recognition means uses a camera to capture an image of the garbage and transmits the data to a server.

[1488] The artificial intelligence generating means analyzes the image data received by the server and identifies the type of waste using a self-learning function.

[1489] The identification result is transmitted to the robotics means through the communication means.

[1490] The robotic means operates robotic arms and conveyor belts to transport identified waste to the appropriate sorting zone.

[1491] The interface means provides guidance to the user by voice or display.

[1492] The emotion recognition means recognizes the user's emotions and provides optimal guidance based on the feedback.

[1493] The user feedback processing means collects feedback from users and reflects it in improving the system.

[1494] The self-learning mechanism continuously updates the generative AI model based on collected data, improving recognition accuracy and user experience.

[1495] Specific operation example

[1496] Example of a smart trash can for home use

[1497] 1. The device (the camera built into the smart trash can) captures an image of the plastic bottle.

[1498] 2. The terminal sends the image data to the server.

[1499] 3. The server uses a generative AI model to identify it as a plastic bottle and sends instructions to the smart trash can to sort it into a plastic sorting zone.

[1500] 4. The terminal opens the plastic lid based on the instructions and guides the user.

[1501] 5. The device analyzes the user's emotions using an emotion engine, and if stress is sensed, it changes the voice tone to provide guidance to reassure the user.

[1502] Example of a robotics system for an incineration plant

[1503] 1. The terminal (a camera on the conveyor belt at the waste disposal plant) captures images of mixed waste.

[1504] 2. The terminal sends the image data to the server.

[1505] 3. The server uses a generative AI model to identify each type of garbage.

[1506] 4. The server sends each identification result to the robotics system.

[1507] 5. The robotic means (robot arm) transports each waste to the appropriate sorting zone according to the identification results.

[1508] 6. The server uses an emotion engine to collect workers' emotional data and monitor their fatigue and stress levels.

[1509] Prompt Sentence Examples

[1510] Prompt for sorting waste with a smart trash can for home:

[1511] Describe what happens to a plastic bottle after it's dropped into a smart trash can. The system includes a camera, a server, robotics tools, and an emotion engine.

[1512] Prompt for sorting garbage at an incineration plant:

[1513] Describe what happens to the mixed waste after it's placed on the conveyor belt. The system includes a camera, a server, robotics tools, and an emotion engine.

[1514] In this way, the system of the present invention not only improves the efficiency of waste sorting at homes and waste disposal plants and achieves high-precision recycling, but also provides a comfortable experience that takes user emotions into consideration.

[1515] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1516] Step 1:

[1517] The device (the built-in camera of the smart trash can) automatically captures images when trash is dropped in. The input is a physical image of the trash, which is captured by the camera and converted into image data in JPEG format. The camera uses a high-resolution camera (e.g., Logitech C920) to adjust the exposure and focus in real time.

[1518] Step 2:

[1519] The device sends the acquired image data to the server via the Wi-Fi module. The input is the JPEG image data acquired in step 1, which is encrypted and sent. The data is sent securely using TLS (Transport Layer Security).

[1520] Step 3:

[1521] The server inputs the received image data into the generative AI model. The input is JPEG image data, which is input to a virtual machine running TensorFlow on the server. The generative AI model analyzes the image data and performs data calculations to identify the type of garbage. The output is the result of identifying the type of garbage.

[1522] Step 4:

[1523] The server converts the identification results into JSON format and sends them back to the device via a communication method. The input is the identification results from the TensorFlow model, which are serialized into JSON format. The output is the JSON-formatted identification results, which are sent via the secure WebSocket protocol.

[1524] Step 5:

[1525] The terminal (robotics system) operates the robotic arm and conveyor belt based on the identification results. The input is the identification results in JSON format from the server, which is analyzed to generate specific operation instructions. For example, it operates an ABB IRB 6700 robotic arm to perform a specific operation, such as transporting aluminum cans to a metal sorting zone. The output is that the waste is transported to the appropriate sorting zone.

[1526] Step 6:

[1527] The device provides the user with the sorting results and instructions through voice guidance and a display. The inputs are the JSON formatted identification results and text for the voice guidance, which are converted into speech using speech synthesis technology (e.g., Amazon Polly). The output is a message saying, "This garbage is a plastic bottle. Please use a plastic recycling lid."

[1528] Step 7:

[1529] The device collects the user's emotional data and sends it to the server. The input includes facial expressions and voices while the user is performing the classification activity, and data is collected using an facial recognition camera and microphone. This data is analyzed using Microsoft's Emotion API and converted into emotional data. The output is the analyzed emotional data, which is sent to the server via secure communication.

[1530] Step 8:

[1531] The server updates the generative AI model based on emotion data and user feedback. The inputs are emotion data obtained from the Emotion API and user feedback data, which are analyzed to adjust the parameters of the generative AI model. The data is then processed to retrain the model's neural network and improve its recognition accuracy. The output is an updated generative AI model, which improves recognition accuracy and the user experience from the next time onwards.

[1532] (Application example 2)

[1533] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1534] Traditionally, parts sorting and quality inspection in factories have been largely manual and inefficient. Furthermore, workers' stress and emotional states have not been properly managed, leading to a deterioration in the working environment. This has led to reduced production efficiency and increased likelihood of work errors, affecting overall work efficiency. To solve these problems, a system is needed that can automatically identify and sort parts while simultaneously managing the emotional states of workers.

[1535] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1536] In this invention, the server includes image recognition means, generative artificial intelligence means, robotics means, communication means, interface means, emotion recognition means, and emotion-based feedback means, thereby improving the efficiency of automatic part identification and sorting, and enabling real-time monitoring of the emotional state of workers and appropriate feedback.

[1537] An "image recognition means" is a device that uses a camera or sensor to capture an image of an object, analyzes the data, and identifies the type and condition of the object.

[1538] The "generative artificial intelligence means" is an artificial intelligence system that identifies the target object based on the acquired image data and improves the accuracy of identification through its self-learning function.

[1539] "Robotics means" refers to devices such as robotic arms and conveyor belts that transport identified objects to the appropriate location.

[1540] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[1541] "Interface means" refers to an operation panel or software application that facilitates user interaction with the system.

[1542] "Emotion recognition means" refers to a device or software that analyzes a user's facial image to determine the user's emotional state.

[1543] The "means for providing feedback based on emotions" refers to a device or software that provides appropriate guidance or feedback to the user based on the results of emotion recognition.

[1544] This invention is a system for improving the efficiency of parts sorting and quality inspection in factories. It also monitors the emotional state of workers in real time and manages their stress. The specific system configuration, usage, and operational procedures are described below.

[1545] 1. System Configuration

[1546] The system includes the following main elements:

[1547] Image recognition means: The camera in the smart glasses captures an image of the part.

[1548] Generative AI means: Analyzes acquired image data and identifies the type and quality of parts. For example, it has the ability to determine whether a product is good or defective.

[1549] Robotic means: Robotic arms and conveyor belts to transport identified parts to the appropriate location.

[1550] Communications: The network and communications protocols used to send and receive data to and from servers and other devices.

[1551] Interface means: An operation panel or software application that allows the worker to interact with the system. This can be the display or voice guidance on the smart glasses.

[1552] Emotion recognition means: Software that captures facial images of workers using the camera in the smart glasses and analyzes their emotional state.

[1553] Means of providing emotion-based feedback: Analyze the emotional state and provide feedback based on stress, fatigue, etc. via audio guidance, etc.

[1554] 2. Program Processing

[1555] Image Acquisition and Recognition

[1556] The device (the camera in the smart glasses) captures an image of the part. Image recognition means is used to analyze the image data and identify the type and condition (good or bad) of the part.

[1557] The server sends the image data to a generative AI model for part identification, which automatically identifies the part type and quality.

[1558] Emotion Recognition and Feedback

[1559] The device (the camera in the smart glasses) captures facial images of the worker, which are then analyzed by emotion recognition means to determine the worker's emotional state (especially their stress level).

[1560] Based on the results of emotion recognition, the server analyzes the data using generative artificial intelligence and provides appropriate guidance to the worker through emotion-based feedback, such as "take a break" if the worker is feeling high stress.

[1561] As a concrete example, imagine a scene where a worker is wearing smart glasses and working on a factory production line. Every time a part comes through, the camera in the smart glasses takes an image of the part, and the AI ​​immediately identifies whether it is a good or defective part. If a defective part is detected, a voice message will be played saying, "A defective part has been detected. Please remove it." The system will also analyze the worker's facial expression, and if the worker's stress level is high, a voice message will be played saying, "You are under a lot of stress. Please take a short break."

[1562] 3. Example prompts

[1563] Input image: An image of a part on a factory production line

[1564] Task: Identify whether the part is good or bad.

[1565] Input image: Face image of a worker

[1566] Task: Identify emotions and stress levels from workers' facial expressions.

[1567] This system will significantly improve the efficiency of automatic identification and sorting of parts in factories, and by monitoring the emotional state of workers and providing appropriate feedback, it will improve the working environment and increase production efficiency.

[1568] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1569] Step 1:

[1570] The device uses the camera in the smart glasses to capture images of parts on the factory production line. The input is image data of the parts, and the output is an image file.

[1571] Step 2:

[1572] The terminal transmits the acquired image data to the server using a communication means. The input is the image file, and the output is the result of the data transmission to the server.

[1573] Step 3:

[1574] The server inputs the received image data into the artificial intelligence generating means to identify the type and quality of the part. The input is the image file, and the output is the label of the part's identification result (e.g., good, defective).

[1575] Step 4:

[1576] The server sends the identification results to the robotics means and issues instructions for automatic sorting. The input is the label of the part identification result, and the output is operation instructions for the robot arm and conveyor belt.

[1577] Step 5:

[1578] The terminal acquires the facial image of the worker using the camera in the smart glasses. The input is the facial image data, and the output is a facial image file.

[1579] Step 6:

[1580] The terminal transmits the facial image data to the server using a communication means. The input is a facial image file, and the output is the result of the data transmission to the server.

[1581] Step 7:

[1582] The server inputs the received facial image data into the emotion recognition means to determine the worker's emotional state and stress level. The input is a facial image file, and the output is a label of the emotional state and a numerical value of the stress level.

[1583] Step 8:

[1584] The server generates a feedback message based on the emotional state and stress level using a generation artificial intelligence means and transmits it to the terminal, where the input is the label of the emotional state and the numerical value of the stress level, and the output is the feedback message.

[1585] Step 9:

[1586] The terminal provides the worker with a feedback message through an interface means such as voice guidance or display, etc. The input is the feedback message, and the output is the voice or display.

[1587] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1588] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1589] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1590] [Fourth embodiment]

[1591] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1592] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1593] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1594] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1595] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1596] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1597] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1598] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1599] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1600] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1601] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1602] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1603] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1604] 1. System Overview

[1605] The present invention is a system for automatically separating waste in homes and waste disposal plants, thereby improving recycling rates. The system includes image recognition means, generative artificial intelligence means, robotics means, communication means, and interface means, which work together to make waste separation more efficient.

[1606] 2. System Configuration

[1607] Image recognition method: Images of garbage are captured using cameras and sensors, and the data is analyzed to identify the type of garbage.

[1608] Generative AI means: Identifies the type of garbage using acquired image data, and improves accuracy through self-learning functions.

[1609] Robotic means: Devices such as robotic arms and conveyor belts that transport identified waste to the appropriate sorting zone.

[1610] Communications: The network and communications protocols used to send and receive data to and from servers and other devices.

[1611] Interface means: An operating panel or software application that facilitates user interaction with the system.

[1612] 3. Program Processing and Operation

[1613] Garbage image acquisition and recognition

[1614] 1. The device (e.g., a camera mounted on a smart trash can or a garbage truck) captures an image every time garbage is dropped in and sends the image data to a server.

[1615] 2. The server inputs the received image data into a generative AI model to identify the type of garbage.

[1616] 3. The server sends the identification result to the robotic means.

[1617] Sorting garbage

[1618] 4. Based on the identification results from the server, the terminal (robotics system) operates the robotic arm and conveyor belt to transport the waste to the appropriate sorting zone.

[1619] Home and waste incineration plant systems

[1620] 5. The terminal will provide voice guidance and display the type of garbage, and guide the user on the appropriate sorting method.

[1621] 6. Users can also use a smartphone app to take photos of trash and send them to the server, which then notifies the user of the image identification results and sorting instructions.

[1622] 4. Specific Examples

[1623] Example 1: Smart trash can for home use

[1624] The device (a camera built into the smart trash can) captures an image of the plastic bottle.

[1625] The terminal transmits the image data to the server.

[1626] The server uses generative AI to identify it as a plastic bottle and sends instructions to the smart trash can to sort it into a plastic sorting zone.

[1627] The terminal opens the plastic lid based on the instructions and guides the user.

[1628] Example 2: Robotics system for waste incineration plants

[1629] The terminal (a camera on the conveyor belt at the waste disposal plant) captures images of the mixed waste.

[1630] The terminal transmits the image data to the server.

[1631] The server uses a generative AI to identify each type of garbage.

[1632] The server sends each identification result to the robotics system.

[1633] A robotic means (robot arm) transports each waste to the appropriate sorting zone according to the identification results.

[1634] 5. Feedback and self-study

[1635] Users can send feedback through the smart trash can or smartphone app, and if the sorting is incorrect, the user's feedback will be reflected in the system.

[1636] The server collects feedback data and updates the generative AI model, which continuously improves its classification accuracy.

[1637] In this way, the system of the present invention improves the efficiency of waste sorting at homes and waste disposal plants, and realizes highly accurate recycling.

[1638] The processing flow will be explained below.

[1639] Processing flow for smart household trash cans

[1640] Step 1: Image capture of the garbage

[1641] Operation details:

[1642] The device (a camera built into the smart trash can) takes an image every time trash is dropped in.

[1643] Step 2: Sending images

[1644] Operation details:

[1645] The terminal transmits the captured image data to the server.

[1646] Step 3: Image Identification (using generative AI)

[1647] Operation details:

[1648] The server inputs the received image data into a generative AI model to identify the type of garbage.

[1649] Step 4: Notification of identification results

[1650] Operation details:

[1651] The server transmits the identification result to the terminal.

[1652] Step 5: Providing guidance on waste sorting

[1653] Operation details:

[1654] The terminal opens the lid of the appropriate sorting zone based on the type of waste identified.

[1655] The terminal guides the user on how to separate the waste by voice guidance or display on the screen.

[1656] Processing flow in a robotics system for waste incineration plants

[1657] Step 1: Image capture of the garbage

[1658] Operation details:

[1659] The terminal (a camera on the conveyor belt at the waste disposal plant) takes images of the waste as it flows by.

[1660] Step 2: Sending images

[1661] Operation details:

[1662] The terminal transmits the captured image data to the server.

[1663] Step 3: Image Identification (using generative AI)

[1664] Operation details:

[1665] The server inputs the received image data into a generative AI model to identify each type of garbage.

[1666] Step 4: Notification of identification results

[1667] Operation details:

[1668] The server sends each identification result to the robotics system.

[1669] Step 5: Sorting your trash

[1670] Operation details:

[1671] The terminal (robot arm) transports each piece of garbage to the appropriate sorting zone based on the identification results from the server.

[1672] Processing flow for smartphone app integration

[1673] Step 1: Photograph the litter

[1674] Operation details:

[1675] Users take photos of the trash using a smartphone app.

[1676] Step 2: Send your photos

[1677] Operation details:

[1678] The terminal transmits the photographed photo data to the server.

[1679] Step 3: Image Identification (using generative AI)

[1680] Operation details:

[1681] The server inputs the received image data into a generative AI model to identify the type of garbage.

[1682] Step 4: Propose a sorting method

[1683] Operation details:

[1684] Based on the identification results, the server sends the appropriate sorting method to the terminal (smartphone app).

[1685] The terminal displays the sorting method to the user.

[1686] Feedback and self-learning process flow

[1687] Step 1: Submit your feedback

[1688] Operation details:

[1689] Users can provide feedback on their sorting results via a smartphone app or smart trash can.

[1690] Step 2: Receive and analyze feedback

[1691] Operation details:

[1692] The server analyzes the received feedback data.

[1693] Step 3: Update the AI ​​model

[1694] Operation details:

[1695] The server updates the generative AI model based on the feedback data.

[1696] Step 4: Apply the model

[1697] Operation details:

[1698] The server will use the updated AI model and apply it to future image recognition processes.

[1699] Through these processing steps, the sorting accuracy of the entire system will improve, aiming to increase recycling rates and realize a sustainable society.

[1700] Example 1

[1701] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1702] Conventional waste sorting at homes and waste disposal plants relies on manual labor or simple machines, resulting in low sorting accuracy and limiting the improvement of recycling rates. Manual sorting is also labor-intensive and time-consuming, and has many issues with efficiency. Furthermore, when incorrect sorting occurs, it is difficult to incorporate user feedback into the system, and the system lacks a self-learning function. There is a need for a system that can solve these problems and achieve high-precision, efficient waste sorting.

[1703] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1704] In this invention, the server includes a means for transmitting image data from the terminal to the server, a means for analyzing image data in the server, and a means for transmitting identification results from the server to the robotics means. This enables highly accurate identification of waste types. The server also includes a means for receiving feedback data from users and a means for self-learning and updating the generative artificial intelligence model, allowing for continuous improvement in identification accuracy through feedback. Furthermore, the robotics means automatically transports waste to the sorting zone, significantly improving work efficiency. This system can be used for both home and waste disposal plants, contributing to increased recycling rates.

[1705] "Image recognition means" refers to a device or software that analyzes acquired image data and identifies the type of garbage.

[1706] "Generative artificial intelligence means" refers to an AI model and related system that uses acquired image data to identify the type of garbage and improves identification accuracy through self-learning functions.

[1707] "Robotics means" refers to devices such as robotic arms and conveyor belts that transport identified waste to the appropriate sorting zone.

[1708] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[1709] "Interface means" refers to an operation panel or software application that facilitates user interaction with the system.

[1710] The "means for transmitting image data from the terminal to the server" is a function for transmitting image data captured by the terminal to the server in real time.

[1711] "Image data analysis means on the server" is a function that inputs the image data received by the server into a generative AI model, analyzes it, and identifies the type of garbage.

[1712] The "means for transmitting identification results from the server to the robotics means" is a function for transmitting the identification results analyzed by the server to the robotics means.

[1713] The "means for receiving feedback data from users" is a function that receives feedback data provided by users via smartphone apps or the like on the server.

[1714] The "self-learning update means for the generative AI model" is a function that updates the learning of the generative AI model using received feedback data to improve the recognition accuracy.

[1715] System Overview

[1716] The present invention is a system for automatically separating waste at homes and waste disposal plants to improve recycling rates. This system operates by linking together image recognition means, generative artificial intelligence means, robotics means, communication means, and interface means.

[1717] Hardware and software used

[1718] Image recognition method: Images of garbage are captured using cameras or sensors, and the data is analyzed. Specifically, a high-resolution camera (e.g., Sony's IMX577 sensor) is used.

[1719] Generative AI method: The acquired image data is input into a generative AI model (e.g., a deep learning model using TensorFlow or PyTorch) to identify the type of garbage.

[1720] Robotic means: using robotic arms or conveyor belts that transport identified waste to the appropriate sorting zone. Examples include robotic arms from KUKA and Fanuc.

[1721] Communication method: Uses a network (e.g., Wi-Fi module and communication protocols such as HTTP, WebSocket, and MQTT) to send and receive data to and from servers and other devices.

[1722] Interface means: An operation panel or software application that makes it easy for the user to interact with the system. Examples include Android and iOS applications.

[1723] Image acquisition and identification of debris

[1724] 1. The device uses cameras installed in smart trash bins or on conveyor belts at waste disposal plants to capture images every time trash is dropped in.

[1725] 2. The device transmits the acquired image data to the server in real time.

[1726] 3. The server inputs the received image data into a generative AI model to identify the type of garbage. Because the generative AI model has been pre-trained on a large dataset of garbage images, it is able to identify the type of garbage with high accuracy.

[1727] 4. The server sends the identification result to the robotic means.

[1728] Sorting garbage

[1729] 5. The terminal (robotics system) receives the identification results from the server and begins to act based on that information. Specifically, the robotic arm moves and transports the identified waste to the appropriate sorting zone.

[1730] Display of sorting instructions

[1731] 6. The terminal provides the user with guidance on proper waste separation, using a display and a voice guidance system.

[1732] User Feedback

[1733] 7. Users send feedback to the system through a smartphone app. For example, if a garbage item is misclassified, they can send feedback from the app, such as, "This garbage is not a plastic bottle, but an aluminum can."

[1734] Learning from feedback data

[1735] 8. The server aggregates user feedback data and retrains the generative AI model to continuously improve its classification accuracy, periodically updating the model using online learning techniques.

[1736] Specific examples

[1737] Example 1: Smart trash can for home use

[1738] The device (a camera built into the smart trash can) captures an image of the plastic bottle.

[1739] The terminal transmits this image data to the server.

[1740] The server uses a generative AI model to identify the bottle as a PET bottle and sort it into a separate zone for plastics.

[1741] The terminal opens the plastic lid and guides the user.

[1742] Example 2: Robotics system for waste incineration plants

[1743] The terminal (a camera on the conveyor belt of the waste disposal plant) captures images of the mixed waste.

[1744] The terminal transmits the image data to the server.

[1745] The server uses a generative AI model to identify each type of garbage.

[1746] The server sends each identification result to the robotics system.

[1747] A robotic means (robot arm) transports each waste to the appropriate sorting zone according to the identification results.

[1748] Prompt Sentence Examples

[1749] "What is this garbage?"

[1750] "Please identify the trash in this image."

[1751] "Which sorting zone should this garbage be sorted into?"

[1752] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1753] Step 1:

[1754] The device uses a camera to capture images of the garbage. The device uses a high-resolution camera to capture pixel information. The captured image data is temporarily stored in the camera's memory.

[1755] Input: Physical presence of litter

[1756] Data processing: Image capture using a high-resolution camera

[1757] Output: High-resolution image data

[1758] Step 2:

[1759] The image data acquired by the device is sent to the server in real time using Wi-Fi or other communication protocols (e.g., HTTP or WebSocket).

[1760] Input: High-resolution image data

[1761] Data processing: Converting image data into digital format and sending it

[1762] Output: Image data sent to the server

[1763] Step 3:

[1764] The server inputs the received image data into a generative AI model for data analysis. The generative AI model uses deep learning algorithms such as TensorFlow and PyTorch. The model analyzes the image data and identifies the type of garbage.

[1765] Input: Image data

[1766] Data processing / calculation: Image analysis and garbage type identification using generative AI models

[1767] Output: Classification result data (e.g. "PET bottle")

[1768] Step 4:

[1769] The server transmits the identification results to the robotic means using a lightweight communication protocol such as MQTT, which allows for low-latency communication.

[1770] Input: Classification result data

[1771] Data processing: Convert the identification result data into digital format and send it

[1772] Output: Identification result data sent to robotic means

[1773] Step 5:

[1774] The terminal (robotics system) receives the identification results from the server and starts the waste sorting process based on them. Specifically, the robotic arm moves and transports the identified waste to the appropriate sorting zone. The robotic arm used could be a product from KUKA or Fanuc, for example.

[1775] Input: Classification result data

[1776] Data processing: Controlling robot behavior based on classification results

[1777] Output: Execution of waste sorting work (transfer of waste)

[1778] Step 6:

[1779] The terminal provides users with guidance on how to separate their waste. It uses a display and a voice guidance system to provide specific instructions on how to separate waste. The terminal's display uses a highly visible LCD panel, and voice information is provided through a speaker.

[1780] Input: Classification result data

[1781] Data processing: Creating content for waste sorting guidance

[1782] Output: Display or audio guidance for sorting

[1783] Step 7:

[1784] Users can provide feedback through a smartphone app. For example, if a garbage item is misclassified, they can provide feedback such as, "This garbage is not a plastic bottle, but an aluminum can." The app is developed using React Native and Flutter.

[1785] Input: User feedback information

[1786] Data processing: digitization and transmission of feedback information

[1787] Output: Feedback data sent to the server

[1788] Step 8:

[1789] The server collects the received feedback data and uses it to retrain the generative AI model. The feedback data is added to the model training dataset to improve classification accuracy. The model is periodically updated using online learning techniques.

[1790] Input: User feedback data

[1791] Data processing / calculation: Aggregating and generating feedback information Retraining AI models

[1792] Output: An updated generative AI model

[1793] Through these steps, the system automatically separates waste with high accuracy and efficiency, improving recycling rates.

[1794] (Application example 1)

[1795] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1796] Conventional waste sorting systems are primarily used in households and waste disposal plants, making it difficult to efficiently separate waste generated within the plant. Issues also existed, such as problems with identification accuracy and reduced recycling efficiency due to incorrect sorting. Furthermore, the system's self-learning capabilities were insufficient to improve accuracy, and effective use of user feedback was also required.

[1797] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1798] In this invention, the server includes an image acquisition means, a recognition means using a generative AI model, a robot control means, a communication means, and an interface means. This enables automatic and highly accurate waste sorting within the factory. Specifically, efficient waste sorting is achieved by coordinating image acquisition from a camera device, waste type identification using a generative AI model, and waste transportation by a robot arm. In addition, a self-learning function continuously improves the accuracy of identification, and by incorporating user feedback as data, even more accurate recycling is possible.

[1799] "Image acquisition means" refers to a device or sensor used to acquire image data from an object.

[1800] "Recognition means using generative AI models" refers to a method of using artificial intelligence to analyze acquired image data and identify the type of object.

[1801] "Robot control means" refers to a system or device that controls an automatically operating robot to perform a specific action or task.

[1802] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[1803] "Interface means" refers to devices and programs such as operation panels and software applications that make it easier for users to operate the system.

[1804] "Industrial waste sorting equipment" means equipment for automatically identifying waste generated within a factory and transporting it to the appropriate sorting zone.

[1805] "Artificial intelligence with self-learning capabilities" refers to artificial intelligence that has the ability to continuously improve its recognition accuracy based on acquired data and user feedback.

[1806] This invention relates to a smart factory waste sorting robot system that automatically sorts waste generated in factories and improves recycling efficiency. The system includes an image acquisition means, a recognition means using a generative AI model, a robot control means, a communication means, an interface means, an industrial waste sorting device, and an artificial intelligence with a self-learning function, which work together to improve the efficiency of waste sorting.

[1807] The image acquisition means provided by the server acquires image data of waste using a camera device. Examples of hardware used include a camera device such as the Logitech C920. This camera device is installed on the conveyor belt where waste is mixed in the factory and takes images periodically.

[1808] The server inputs the acquired image data into a recognition means using a generative AI model to identify the type of waste. This generative AI model is built using TensorFlow Keras and performs highly accurate waste identification. The server then sends the identification results to the robot control means, which provides instructions for transporting the waste to the appropriate sorting zone.

[1809] The robot control means provided by the terminal operates the robot arm based on instructions from the server and transports the waste to the appropriate sorting zone. An example of the robot arm used is the UR5 robot arm. The robot control means is also equipped with an industrial waste sorting device, enabling efficient transport of waste.

[1810] The communication means includes the network and communication protocols for transmitting and receiving data between the server, terminal, and other devices, which allows the entire system to work together.

[1811] The interface means includes an operation panel and software applications that allow users to easily operate the system, allowing them to provide feedback to the system and contribute to improving its accuracy. Furthermore, the system has a self-learning function that allows it to continuously improve its classification accuracy based on user feedback.

[1812] As a concrete example, imagine a conveyor belt with mixed waste. A camera on this conveyor belt periodically takes images and sends them to a generative AI model for identification. For example, use the following prompt:

[1813] Input image: (uploaded garbage image)

[1814] ---

[1815] Question: What type of trash is in this image?

[1816] ---

[1817] Example output:

[1818] plastic

[1819] metal

[1820] General waste

[1821] As described above, the present invention enables waste materials in a factory to be separated efficiently and with high accuracy, thereby improving recycling efficiency.

[1822] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1823] Step 1:

[1824] The terminal uses a camera device to capture images of the waste on the conveyor belt. The input is image data from the camera device and the output is the captured image. This image data is sent to the next processing step.

[1825] Step 2:

[1826] The device sends the acquired image data to the server. The input is image data from the camera device, and the output is the image data sent to the server. The server receives this data and analyzes it using a generative AI model.

[1827] Step 3:

[1828] The server inputs the image data into a generative AI model to identify the type of waste. The input is the submitted image data, and the output is the identified waste type (e.g., plastic, metal, general waste). The identification result is sent to the next processing step.

[1829] Step 4:

[1830] The server transmits the identification result to the robot control means, where the input is the identification result regarding the type of waste, and the output is instruction data for the robot control means, including instructions for transporting the waste to the appropriate sorting zone.

[1831] Step 5:

[1832] The terminal controls the robot arm to transport the identified waste to the appropriate sorting zone. The input is instruction data from the server, and the output is the waste transported to the appropriate sorting zone. The robot arm operates based on the identification result.

[1833] Step 6:

[1834] The user operates the system using an interface means and provides feedback as needed. The input is the user's operation and feedback data, and the output is the feedback information sent to the system. This feedback information is collected by the server and used to update the self-learning artificial intelligence.

[1835] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1836] 1. System Overview

[1837] The present invention is a system for automatically sorting waste in homes and waste disposal plants to improve recycling rates. In particular, this system features an emotion engine that recognizes the user's emotions, providing more appropriate feedback and guidance. The system includes image recognition means, generative artificial intelligence means, robotics means, communication means, interface means, and the emotion engine, which work together to make waste sorting more efficient.

[1838] 2. System Configuration

[1839] Image recognition method: Images of garbage are captured using cameras and sensors, and the data is analyzed to identify the type of garbage.

[1840] Generative AI means: Identifies the type of garbage using acquired image data, and improves accuracy through self-learning functions.

[1841] Robotic means: Devices such as robotic arms and conveyor belts that transport identified waste to the appropriate sorting zone.

[1842] Communications: The network and communications protocols used to send and receive data to and from servers and other devices.

[1843] Interface means: An operating panel or software application that facilitates user interaction with the system.

[1844] Emotion engine: Technology that recognizes user emotions and optimizes feedback and guidance based on those emotions.

[1845] 3. Program Processing and Operation

[1846] Garbage image acquisition and recognition

[1847] 1. The device (e.g., a camera mounted on a smart trash can or a garbage truck) captures an image every time garbage is dropped in and sends the image data to a server.

[1848] 2. The server inputs the received image data into a generative AI model to identify the type of garbage.

[1849] 3. The server sends the identification result to the robotic means.

[1850] Sorting garbage

[1851] 4. Based on the identification results from the server, the terminal (robotics system) operates the robotic arm and conveyor belt to transport the waste to the appropriate sorting zone.

[1852] Home and waste incineration plant systems

[1853] 5. The terminal will provide voice guidance and display the type of garbage, and guide the user on the appropriate sorting method.

[1854] 6. Users can also use a smartphone app to take photos of trash and send them to the server, which then notifies the user of the image identification results and sorting instructions.

[1855] Emotion Recognition and Feedback

[1856] 7. The user's emotion data collected on the terminal is sent to the server via communication means.

[1857] 8. The server uses an emotion engine to recognize the user's emotional state and collect it as feedback data.

[1858] 9. The server updates the generative artificial intelligence means based on the emotion data to improve the recognition accuracy and user experience of the entire system.

[1859] 4. Specific Examples

[1860] Example 1: Smart trash can for home use

[1861] The device (a camera built into the smart trash can) captures an image of the plastic bottle.

[1862] The terminal transmits the image data to the server.

[1863] The server uses generative AI to identify it as a plastic bottle and sends instructions to the smart trash can to sort it into a plastic sorting zone.

[1864] The terminal opens the plastic lid based on the instructions and guides the user.

[1865] The device analyzes the user's emotions using an emotion engine, and if stress is felt, it changes the tone of the voice to provide guidance to reassure the user.

[1866] Example 2: Robotics system for waste incineration plants

[1867] The terminal (a camera on the conveyor belt at the waste disposal plant) captures images of the mixed waste.

[1868] The terminal transmits the image data to the server.

[1869] The server uses a generative AI to identify each type of garbage.

[1870] The server sends each identification result to the robotics system.

[1871] A robotic means (robot arm) transports each waste to the appropriate sorting zone according to the identification results.

[1872] The server uses an emotion engine to collect workers' emotional data and monitor their fatigue and stress levels.

[1873] 5. Feedback and self-learning

[1874] Users can send feedback through the smart trash can or smartphone app, and if the sorting is incorrect, the user's feedback will be reflected in the system.

[1875] The server collects feedback and emotion data and updates the generative AI model, which continuously improves its classification accuracy.

[1876] In this way, the system of the present invention not only improves the efficiency of waste sorting at homes and waste disposal plants and achieves highly accurate recycling, but also provides a comfortable experience that takes the user's emotions into consideration.

[1877] The processing flow will be explained below.

[1878] Processing flow for smart household trash cans

[1879] Step 1: Image capture of the garbage

[1880] Operation details:

[1881] The device (a camera built into the smart trash can) takes an image every time trash is dropped in.

[1882] Step 2: Sending images

[1883] Operation details:

[1884] The terminal transmits the captured image data to the server.

[1885] Step 3: Image Identification (using generative AI)

[1886] Operation details:

[1887] The server inputs the received image data into a generative AI model to identify the type of garbage.

[1888] Step 4: Notification of identification results

[1889] Operation details:

[1890] The server transmits the identification result to the terminal.

[1891] Step 5: Providing guidance on waste sorting

[1892] Operation details:

[1893] The terminal opens the lid of the appropriate sorting zone based on the type of waste identified.

[1894] The terminal guides the user on how to separate the waste by voice guidance or display on the screen.

[1895] Step 6: Collect emotion data

[1896] Operation details:

[1897] The terminal uses an emotion engine to collect the user's voice tone and movement data.

[1898] Step 7: Sending Emotion Data

[1899] Operation details:

[1900] The terminal transmits the collected emotion data to the server.

[1901] Step 8: Analyze the sentiment data

[1902] Operation details:

[1903] The server uses an emotion engine to analyze the user's emotional state and generate feedback based thereon.

[1904] Step 9: View your feedback

[1905] Operation details:

[1906] The server transmits feedback based on the analysis results to the terminal.

[1907] The terminal provides the user with a steady tone and thoughtful guidance.

[1908] Processing flow in a robotics system for waste incineration plants

[1909] Step 1: Image capture of the garbage

[1910] Operation details:

[1911] The terminal (a camera on the conveyor belt at the waste disposal plant) takes images of the waste as it flows by.

[1912] Step 2: Sending images

[1913] Operation details:

[1914] The terminal transmits the captured image data to the server.

[1915] Step 3: Image Identification (using generative AI)

[1916] Operation details:

[1917] The server inputs the received image data into a generative AI model to identify each type of garbage.

[1918] Step 4: Notification of identification results

[1919] Operation details:

[1920] The server sends each identification result to the robotics system.

[1921] Step 5: Sorting your trash

[1922] Operation details:

[1923] The terminal (robot arm) transports each piece of garbage to the appropriate sorting zone based on the identification results from the server.

[1924] Step 6: Collect worker emotion data

[1925] Operation details:

[1926] The device uses an emotion engine to analyze the worker's movements and voice and collect emotional data.

[1927] Step 7: Sending Emotion Data

[1928] Operation details:

[1929] The terminal transmits the collected emotion data to the server.

[1930] Step 8: Worker condition monitoring

[1931] Operation details:

[1932] The server uses an emotion engine to monitor workers' stress levels and fatigue and generate feedback.

[1933] Step 9: Provide monitoring results

[1934] Operation details:

[1935] The server sends the monitoring results to the terminal and notifies the workers and managers.

[1936] Processing flow for smartphone app integration

[1937] Step 1: Photograph the litter

[1938] Operation details:

[1939] Users take photos of the trash using a smartphone app.

[1940] Step 2: Send your photos

[1941] Operation details:

[1942] The terminal transmits the photographed photo data to the server.

[1943] Step 3: Image Identification (using generative AI)

[1944] Operation details:

[1945] The server inputs the received photo data into a generative AI model to identify the type of litter.

[1946] Step 4: Propose a sorting method

[1947] Operation details:

[1948] Based on the identification results, the server sends the appropriate sorting method to the terminal (smartphone app).

[1949] The terminal displays the sorting method to the user.

[1950] Step 5: Collecting sentiment data

[1951] Operation details:

[1952] The device analyzes the user's voice and actions using an emotion engine to collect emotion data.

[1953] Step 6: Sending Emotion Data

[1954] Operation details:

[1955] The terminal transmits the collected emotion data to the server.

[1956] Step 7: Analyze the sentiment data

[1957] Operation details:

[1958] The server uses an emotion engine to analyze the user's emotional state and generate feedback.

[1959] Step 8: Viewing feedback

[1960] Operation details:

[1961] The server sends optimized feedback based on the emotion data to the terminal.

[1962] The terminal displays a thoughtful guide to the user.

[1963] Feedback and self-learning process flow

[1964] Step 1: Submit your feedback

[1965] Operation details:

[1966] Users can provide feedback through the smart trash can or through a smartphone app.

[1967] Step 2: Receive and analyze feedback

[1968] Operation details:

[1969] The server analyzes the received feedback data.

[1970] Step 3: Update the AI ​​model

[1971] Operation details:

[1972] The server updates the generative AI model based on the feedback data.

[1973] Step 4: Apply the model

[1974] Operation details:

[1975] The server will use the updated AI model and apply it to future image recognition processes.

[1976] Step 5: Learning with emotion data

[1977] Operation details:

[1978] The server uses the data collected by the emotion engine to self-train the generative AI and provide feedback according to emotions.

[1979] In this way, the system of the present invention not only improves the efficiency of waste sorting at homes and waste disposal plants and achieves highly accurate recycling, but also provides a comfortable experience that takes the user's emotions into consideration.

[1980] Example 2

[1981] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1982] Conventional waste sorting systems have limitations in the accuracy and efficiency of sorting, and also have the problem of being unable to take user emotions and feedback into account. Furthermore, waste sorting systems for households and waste disposal plants have also been plagued by issues of incorrect sorting and low system learning accuracy. This has led to a demand for improved recycling rates and user experience.

[1983] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1984] In this invention, the server includes image recognition means, generative artificial intelligence means, robotics means, communication means, interface means, emotion recognition means, user feedback processing means, and self-learning means, which not only enables accurate and efficient waste sorting but also enables feedback that takes user emotions into account and continuous system learning and improvement.

[1985] "Image recognition means" refers to devices and technologies that use cameras or sensors to capture images of garbage and analyze the data to identify the type of garbage.

[1986] "Generative artificial intelligence means" refers to an artificial intelligence system that analyzes acquired image data, uses self-learning functions to identify the type of garbage, and improves identification accuracy.

[1987] "Robotics means" refers to devices such as robotic arms and conveyor belts used to transport identified waste to the appropriate sorting zone.

[1988] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[1989] "Interface means" refers to an operation panel or software application that facilitates user interaction with the system.

[1990] "Emotion recognition means" refers to technology and devices that recognize a user's emotions and optimize feedback and guidance based on those emotions.

[1991] "User feedback processing means" refers to processes and techniques for collecting feedback from users and using it to improve the system.

[1992] "Self-learning tools" refers to learning algorithms and techniques that use collected data to continuously improve the overall system's recognition accuracy and user experience.

[1993] MODE FOR CARRYING OUT THE INVENTION

[1994] System Overview

[1995] The present invention is a system that automatically separates waste in homes and waste disposal plants to improve recycling rates. In particular, the system features an emotion engine that recognizes the user's emotions, providing more appropriate feedback and guidance. The system includes image recognition means, generative artificial intelligence means, robotics means, communication means, interface means, emotion recognition means, user feedback processing means, and self-learning means, which work together to streamline waste separation.

[1996] Hardware and Software Configuration

[1997] The specific hardware and software used in the system are as follows:

[1998] 1. Devices (e.g., smart trash cans and cameras mounted on garbage trucks)

[1999] Hardware: High-resolution camera (e.g. Logitech C920)

[2000] Communication method: Wi-Fi and LTE module

[2001] Interface: Operation panel, display, microphone (e.g., Amazon Polly for voice guidance)

[2002] Emotion Recognition Technology: Microsoft's Emotion API

[2003] 2. Server

[2004] Hardware: High-performance servers (e.g. AWS EC2)

[2005] Software: Generative AI models (e.g., TensorFlow, PyTorch)

[2006] System Operation and Processing

[2007] The operation of this system is realized as follows.

[2008] The image recognition means uses a camera to capture an image of the garbage and transmits the data to a server.

[2009] The artificial intelligence generating means analyzes the image data received by the server and identifies the type of waste using a self-learning function.

[2010] The identification result is transmitted to the robotics means through the communication means.

[2011] The robotic means operates robotic arms and conveyor belts to transport identified waste to the appropriate sorting zone.

[2012] The interface means provides guidance to the user by voice or display.

[2013] The emotion recognition means recognizes the user's emotions and provides optimal guidance based on the feedback.

[2014] The user feedback processing means collects feedback from users and reflects it in improving the system.

[2015] The self-learning mechanism continuously updates the generative AI model based on collected data, improving recognition accuracy and user experience.

[2016] Specific operation example

[2017] Example of a smart trash can for home use

[2018] 1. The device (the camera built into the smart trash can) captures an image of the plastic bottle.

[2019] 2. The terminal sends the image data to the server.

[2020] 3. The server uses a generative AI model to identify it as a plastic bottle and sends instructions to the smart trash can to sort it into a plastic sorting zone.

[2021] 4. The terminal opens the plastic lid based on the instructions and guides the user.

[2022] 5. The device analyzes the user's emotions using an emotion engine, and if stress is sensed, it changes the voice tone to provide guidance to reassure the user.

[2023] Example of a robotics system for an incineration plant

[2024] 1. The terminal (a camera on the conveyor belt at the waste disposal plant) captures images of mixed waste.

[2025] 2. The terminal sends the image data to the server.

[2026] 3. The server uses a generative AI model to identify each type of garbage.

[2027] 4. The server sends each identification result to the robotics system.

[2028] 5. The robotic means (robot arm) transports each waste to the appropriate sorting zone according to the identification results.

[2029] 6. The server uses an emotion engine to collect workers' emotional data and monitor their fatigue and stress levels.

[2030] Prompt Sentence Examples

[2031] Prompt for sorting waste with a smart trash can for home:

[2032] Describe what happens to a plastic bottle after it's dropped into a smart trash can. The system includes a camera, a server, robotics tools, and an emotion engine.

[2033] Prompt for sorting garbage at an incineration plant:

[2034] Describe what happens to the mixed waste after it's placed on the conveyor belt. The system includes a camera, a server, robotics tools, and an emotion engine.

[2035] In this way, the system of the present invention not only improves the efficiency of waste sorting at homes and waste disposal plants and achieves high-precision recycling, but also provides a comfortable experience that takes user emotions into consideration.

[2036] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2037] Step 1:

[2038] The device (the built-in camera of the smart trash can) automatically captures images when trash is dropped in. The input is a physical image of the trash, which is captured by the camera and converted into image data in JPEG format. The camera uses a high-resolution camera (e.g., Logitech C920) to adjust the exposure and focus in real time.

[2039] Step 2:

[2040] The device sends the acquired image data to the server via the Wi-Fi module. The input is the JPEG image data acquired in step 1, which is encrypted and sent. The data is sent securely using TLS (Transport Layer Security).

[2041] Step 3:

[2042] The server inputs the received image data into the generative AI model. The input is JPEG image data, which is input to a virtual machine running TensorFlow on the server. The generative AI model analyzes the image data and performs data calculations to identify the type of garbage. The output is the result of identifying the type of garbage.

[2043] Step 4:

[2044] The server converts the identification results into JSON format and sends them back to the device via a communication method. The input is the identification results from the TensorFlow model, which are serialized into JSON format. The output is the JSON-formatted identification results, which are sent via the secure WebSocket protocol.

[2045] Step 5:

[2046] The terminal (robotics system) operates the robotic arm and conveyor belt based on the identification results. The input is the identification results in JSON format from the server, which is analyzed to generate specific operation instructions. For example, it operates an ABB IRB 6700 robotic arm to perform a specific operation, such as transporting aluminum cans to a metal sorting zone. The output is that the waste is transported to the appropriate sorting zone.

[2047] Step 6:

[2048] The device provides the user with the sorting results and instructions through voice guidance and a display. The inputs are the JSON formatted identification results and text for the voice guidance, which are converted into speech using speech synthesis technology (e.g., Amazon Polly). The output is a message saying, "This garbage is a plastic bottle. Please use a plastic recycling lid."

[2049] Step 7:

[2050] The device collects the user's emotional data and sends it to the server. The input includes facial expressions and voices while the user is performing the classification activity, and data is collected using an facial recognition camera and microphone. This data is analyzed using Microsoft's Emotion API and converted into emotional data. The output is the analyzed emotional data, which is sent to the server via secure communication.

[2051] Step 8:

[2052] The server updates the generative AI model based on emotion data and user feedback. The inputs are emotion data obtained from the Emotion API and user feedback data, which are analyzed to adjust the parameters of the generative AI model. The data is then processed to retrain the model's neural network and improve its recognition accuracy. The output is an updated generative AI model, which improves recognition accuracy and the user experience from the next time onwards.

[2053] (Application example 2)

[2054] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2055] Traditionally, parts sorting and quality inspection in factories have been largely manual and inefficient. Furthermore, workers' stress and emotional states have not been properly managed, leading to a deterioration in the working environment. This has led to reduced production efficiency and increased likelihood of work errors, affecting overall work efficiency. To solve these problems, a system is needed that can automatically identify and sort parts while simultaneously managing the emotional states of workers.

[2056] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2057] In this invention, the server includes image recognition means, generative artificial intelligence means, robotics means, communication means, interface means, emotion recognition means, and emotion-based feedback means, thereby improving the efficiency of automatic part identification and sorting, and enabling real-time monitoring of the emotional state of workers and appropriate feedback.

[2058] An "image recognition means" is a device that uses a camera or sensor to capture an image of an object, analyzes the data, and identifies the type and condition of the object.

[2059] The "generative artificial intelligence means" is an artificial intelligence system that identifies the target object based on the acquired image data and improves the accuracy of identification through its self-learning function.

[2060] "Robotics means" refers to devices such as robotic arms and conveyor belts that transport identified objects to the appropriate location.

[2061] "Communication means" refers to the network and communication protocols used to send and receive data to and from servers and other devices.

[2062] "Interface means" refers to an operation panel or software application that facilitates user interaction with the system.

[2063] "Emotion recognition means" refers to a device or software that analyzes a user's facial image to determine the user's emotional state.

[2064] The "means for providing feedback based on emotions" refers to a device or software that provides appropriate guidance or feedback to the user based on the results of emotion recognition.

[2065] This invention is a system for improving the efficiency of parts sorting and quality inspection in factories. It also monitors the emotional state of workers in real time and manages their stress. The specific system configuration, usage, and operational procedures are described below.

[2066] 1. System Configuration

[2067] The system includes the following main elements:

[2068] Image recognition means: The camera in the smart glasses captures an image of the part.

[2069] Generative AI means: Analyzes acquired image data and identifies the type and quality of parts. For example, it has the ability to determine whether a product is good or defective.

[2070] Robotic means: Robotic arms and conveyor belts to transport identified parts to the appropriate location.

[2071] Communications: The network and communications protocols used to send and receive data to and from servers and other devices.

[2072] Interface means: An operation panel or software application that allows the worker to interact with the system. This can be the display or voice guidance on the smart glasses.

[2073] Emotion recognition means: Software that captures facial images of workers using the camera in the smart glasses and analyzes their emotional state.

[2074] Means of providing emotion-based feedback: Analyze the emotional state and provide feedback based on stress, fatigue, etc. via audio guidance, etc.

[2075] 2. Program Processing

[2076] Image Acquisition and Recognition

[2077] The device (the camera in the smart glasses) captures an image of the part. Image recognition means is used to analyze the image data and identify the type and condition (good or bad) of the part.

[2078] The server sends the image data to a generative AI model for part identification, which automatically identifies the part type and quality.

[2079] Emotion Recognition and Feedback

[2080] The device (the camera in the smart glasses) captures facial images of the worker, which are then analyzed by emotion recognition means to determine the worker's emotional state (especially their stress level).

[2081] Based on the results of emotion recognition, the server analyzes the data using generative artificial intelligence and provides appropriate guidance to the worker through emotion-based feedback, such as "take a break" if the worker is feeling high stress.

[2082] As a concrete example, imagine a scene where a worker is wearing smart glasses and working on a factory production line. Every time a part comes through, the camera in the smart glasses takes an image of the part, and the AI ​​immediately identifies whether it is a good or defective part. If a defective part is detected, a voice message will be played saying, "A defective part has been detected. Please remove it." The system will also analyze the worker's facial expression, and if the worker's stress level is high, a voice message will be played saying, "You are under a lot of stress. Please take a short break."

[2083] 3. Example prompts

[2084] Input image: An image of a part on a factory production line

[2085] Task: Identify whether the part is good or bad.

[2086] Input image: Face image of a worker

[2087] Task: Identify emotions and stress levels from workers' facial expressions.

[2088] This system will significantly improve the efficiency of automatic identification and sorting of parts in factories, and by monitoring the emotional state of workers and providing appropriate feedback, it will improve the working environment and increase production efficiency.

[2089] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2090] Step 1:

[2091] The device uses the camera in the smart glasses to capture images of parts on the factory production line. The input is image data of the parts, and the output is an image file.

[2092] Step 2:

[2093] The terminal transmits the acquired image data to the server using a communication means. The input is the image file, and the output is the result of the data transmission to the server.

[2094] Step 3:

[2095] The server inputs the received image data into the artificial intelligence generating means to identify the type and quality of the part. The input is the image file, and the output is the label of the part's identification result (e.g., good, defective).

[2096] Step 4:

[2097] The server sends the identification results to the robotics means and issues instructions for automatic sorting. The input is the label of the part identification result, and the output is operation instructions for the robot arm and conveyor belt.

[2098] Step 5:

[2099] The terminal acquires the facial image of the worker using the camera in the smart glasses. The input is the facial image data, and the output is a facial image file.

[2100] Step 6:

[2101] The terminal transmits the facial image data to the server using a communication means. The input is a facial image file, and the output is the result of the data transmission to the server.

[2102] Step 7:

[2103] The server inputs the received facial image data into the emotion recognition means to determine the worker's emotional state and stress level. The input is a facial image file, and the output is a label of the emotional state and a numerical value of the stress level.

[2104] Step 8:

[2105] The server generates a feedback message based on the emotional state and stress level using a generation artificial intelligence means and transmits it to the terminal, where the input is the label of the emotional state and the numerical value of the stress level, and the output is the feedback message.

[2106] Step 9:

[2107] The terminal provides the worker with a feedback message through an interface means such as voice guidance or display, etc. The input is the feedback message, and the output is the voice or display.

[2108] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2109] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2110] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2111] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2112] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2113] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2114] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2115] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2116] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2117] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2118] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2119] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2120] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2121] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2122] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2123] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2124] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2125] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2126] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2127] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2128] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2129] The following is further disclosed regarding the above embodiment.

[2130] (Claim 1)

[2131] Image recognition means;

[2132] Generative artificial intelligence means;

[2133] Robotics means;

[2134] means of communication;

[2135] an interface means;

[2136] A system including:

[2137] (Claim 2)

[2138] Further comprising a household waste sorting device;

[2139] 10. The system of claim 1.

[2140] (Claim 3)

[2141] Further comprising a waste sorting device for an incineration plant;

[2142] 10. The system of claim 1.

[2143] (Claim 4)

[2144] Further, it includes a means of assisting with waste sorting via a smartphone application.

[2145] 10. The system of claim 1.

[2146] (Claim 5)

[2147] Further comprising a garbage identification means using a QR code or an RFID tag;

[2148] 10. The system of claim 1.

[2149] (Claim 6)

[2150] Further including a voice guidance means,

[2151] 10. The system of claim 1.

[2152] (Claim 7)

[2153] further comprising a camera and sensor means mounted on the garbage collection vehicle;

[2154] 10. The system of claim 1.

[2155] (Claim 8)

[2156] Further comprising a self-learning generating artificial intelligence means;

[2157] 10. The system of claim 1.

[2158] (Claim 9)

[2159] and means for updating the generating artificial intelligence means based on feedback from the user.

[2160] 10. The system of claim 1.

[2161] "Example 1"

[2162] (Claim 1)

[2163] Image recognition means;

[2164] Generative artificial intelligence means;

[2165] Robotics means;

[2166] means of communication;

[2167] an interface means;

[2168] means for transmitting image data from the terminal to the server;

[2169] Image data analysis means in the server;

[2170] means for transmitting an identification result from the server to the robotics means;

[2171] means for receiving feedback data from a user to a server;

[2172] A self-learning update means for the generative artificial intelligence model;

[2173] A system including:

[2174] (Claim 2)

[2175] The household waste sorting device further includes a means for displaying sorting instructions to a user.

[2176] 10. The system of claim 1.

[2177] (Claim 3)

[2178] Further comprising a waste sorting device for a waste incineration plant and a waste sorting operation means,

[2179] 10. The system of claim 1.

[2180] "Application Example 1"

[2181] (Claim 1)

[2182] Image acquisition means;

[2183] A recognition method using a generative AI model;

[2184] a robot control means;

[2185] means of communication;

[2186] an interface means;

[2187] A system including:

[2188] (Claim 2)

[2189]

[0023] Further comprising an industrial waste sorting device;

[2190] 10. The system of claim 1.

[2191] (Claim 3)

[2192] It also includes artificial intelligence with self-learning capabilities.

[2193] 10. The system of claim 1.

[2194] "Example 2: Combining Emotion Engines"

[2195] (Claim 1)

[2196] Image recognition means;

[2197] Generative artificial intelligence means;

[2198] Robotics means;

[2199] means of communication;

[2200] an interface means;

[2201] An emotion recognition means;

[2202] user feedback processing means;

[2203] self-learning tools,

[2204] A system including:

[2205] (Claim 2)

[2206] Further comprising a household waste sorting device;

[2207] 10. The system of claim 1.

[2208] (Claim 3)

[2209] Further comprising a waste sorting device for an incineration plant;

[2210] 10. The system of claim 1.

[2211] "Application example 2 when combining emotion engines"

[2212] (Claim 1)

[2213] Image recognition means;

[2214] Generative artificial intelligence means;

[2215] Robotics means;

[2216] means of communication;

[2217] an interface means;

[2218] An emotion recognition means;

[2219] A means of providing emotion-based feedback;

[2220] A system including:

[2221] (Claim 2)

[2222] Further comprising a household waste sorting device;

[2223] 10. The system of claim 1.

[2224] (Claim 3)

[2225] Further comprising a waste sorting device for an incineration plant;

[2226] 10. The system of claim 1.

[2227] (Claim 4)

[2228]

[0023] Further comprising means for recognizing the emotional state of the worker and for stress management;

[2229] 10. The system of claim 1.

[2230] (Claim 5)

[2231] and means for sorting and quality inspecting the parts and providing feedback based on the recognition results.

[2232] 10. The system of claim 1. [Explanation of symbols]

[2233] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. Image recognition means; Generative artificial intelligence means; Robotics means; means of communication; an interface means; A system including:

2. Further comprising a household waste sorting device; The system of claim 1 .

3. Further comprising a waste sorting device for an incineration plant; The system of claim 1 .

4. Further, it includes a means of assisting with waste sorting via a smartphone application. The system of claim 1 .

5. Further comprising a garbage identification means using a QR code or an RFID tag; The system of claim 1 .

6. Further including a voice guidance means, The system of claim 1 .

7. further comprising a camera and sensor means mounted on the garbage collection vehicle; The system of claim 1 .

8. Further comprising a self-learning generating artificial intelligence means; The system of claim 1 .

9. and means for updating the generating artificial intelligence means based on feedback from the user. The system of claim 1 .

Images