System

Abstract

A system is provided.SOLUTION: The system includes a means for taking a photograph of waste by a user, a means for transmitting photograph data to a server, a means for analyzing the photograph data by the server and discriminating the kind of the waste, a means for generating a sorting method recommended on the basis of the kind of the waste discriminated by the server, and a means for transmitting the sorting method generated by the server to a terminal of the user and displaying it.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, proper waste sorting is crucial for environmental protection and maintaining the recycling system. However, many citizens lack sufficient knowledge about how to sort waste, resulting in frequent mistakes in sorting. This problem has a negative impact on the environment and reduces the efficiency of the recycling process. Furthermore, the time required for waste sorting and the lack of appropriate information hinder citizens from sorting properly. Therefore, there is an urgent need to provide a system that allows for easy and rapid waste sorting. [Means for solving the problem]

[0005] In this invention, when a user takes a photo of garbage, the photo data is sent to a server, which analyzes the photo data. The server determines the type of waste based on the analysis results and generates an appropriate sorting method for that type. The generated sorting method is sent to the user's device and displayed. The user can also input questions in real time, and the server generates answers to those questions using natural language processing. Furthermore, when a user views environmental education content and submits the viewing information, the server awards points to the user based on that information and displays the point information. Furthermore, when a user participates in online community activities or events, the system notifies the user of the participation information, and the server awards rewards and badges based on the participation information and displays that information.

[0006] A "terminal" is a device operated by a user, and refers to a device that has functions such as taking pictures, transmitting data, and receiving and displaying data.

[0007] "Server" refers to a computer system that receives data sent from a terminal, analyzes, processes, and returns generated information to the terminal.

[0008] "Waste" refers to items that users throw away and that should be properly separated.

[0009] "Photo data" refers to image data of waste photographed by a user on a terminal.

[0010] "Image analysis" refers to the technical means by which the server processes photographic data to determine the type and material of the waste.

[0011] "Sorting method" refers to the procedures and techniques for appropriate disposal depending on the type of waste.

[0012] "Natural language processing" refers to the technical means of analyzing questions from users, understanding their meaning, and generating answers.

[0013] "Environmental Educational Content" refers to videos, articles, and other media formats that educate users about environmental protection and proper waste disposal practices.

[0014] "Points" refer to a type of reward that is given to users for actions such as using the system and consuming environmental education content.

[0015] "Online community" refers to a digital platform where users can share information about environmental protection and waste sorting and participate in discussions and events.

[0016] "Rewards" refers to incentives such as points or badges awarded to users based on their activities in online communities or their participation in events.

[0017] A "badge" refers to a type of recognition given within an online community to recognize a user's specific actions or achievements. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention is a system that supports users in taking photos of waste and sorting it appropriately. This system is composed of components such as a terminal, a server, and a user.

[0040] System Overview

[0041] 1. Taking photos of trash and sending the data

[0042] Users take photos of waste using their devices (smartphones or tablets), and the devices send the photo data to the server. For example, if a user takes a photo of a plastic bottle, the photo data is sent to the server.

[0043] 2. Image analysis on the server

[0044] The server passes the received photo data to an image analysis algorithm, which determines the type of waste (e.g., plastic bottle, paper, metal, etc.) based on its shape and characteristics. After analysis, the server confirms that the waste is a plastic bottle.

[0045] 3. Presentation of sorting methods

[0046] Based on the analysis results, the server generates a sorting method appropriate for the PET bottles. For example, it generates a message such as "Please separate PET bottles as plastic waste." This message is sent to the terminal and displayed to the user.

[0047] 4. Real-time chat support

[0048] If the user has any further questions about the sorting method, they can use the chat function on their device to input their question. For example, they can input, "How should I dispose of this bottle cap?" The device sends this question to the server, which then analyzes the question using a natural language processing algorithm and generates an answer. The answer generated, "Please sort the cap as burnable waste," is sent to the device and displayed to the user.

[0049] 5. Environmental education and point redemption

[0050] The user views environmental education content (e.g., videos or articles about recycling). The device sends the viewing information to the server. The server confirms that the viewing has been completed and awards points to the user based on that information. The point information is updated in real time on the user's device and displayed. For example, it may say, "10 points awarded."

[0051] 6. Community Participation and Support

[0052] Users join online communities and participate in discussions and events related to environmental protection. The device notifies the server of their participation. The server records the participation information and awards rewards and badges as necessary. This information is also sent to the device and displayed to the user. For example, the server may notify the user that they have earned a recycling master badge.

[0053] Specific examples

[0054] Example 1: Taking photos of garbage and showing how to separate it

[0055] The user takes a photo of a plastic bottle with their device and sends the data to the server. The server recognizes it as a plastic bottle and generates a message saying "Please separate plastic bottles as plastic waste" and sends it to the device. The message is displayed on the user's device, allowing the user to separate the bottle correctly.

[0056] Example 2: Watching environmental education videos and earning points

[0057] The user watches a video about recycling on their device, and after completing the viewing, the information is sent to the server. The server confirms the completion of the viewing and awards the user 10 points. The user's device displays "10 points awarded," encouraging environmental education.

[0058] This is a support system that helps users easily separate their garbage and raise environmental awareness. It is a powerful tool for achieving sustainable living by providing real-time advice, reward points, and encouraging community participation.

[0059] The processing flow will be explained below.

[0060] Step 1:

[0061] The user launches the camera app on the device and takes a photo of the waste (e.g., a plastic bottle).

[0062] Step 2:

[0063] The device temporarily stores the captured photo data and prepares an HTTP POST request to send it to the server.

[0064] Step 3:

[0065] The device sends the photo data to the server via an HTTP POST request.

[0066] Step 4:

[0067] The server receives the HTTP POST request and retrieves the photo data.

[0068] Step 5:

[0069] The server uses an AI model for image analysis to analyze the photo data and determine the type of waste, using feature extraction and identification algorithms.

[0070] Step 6:

[0071] Based on the analysis results, the server generates an appropriate method for separating waste according to the type of waste. For example, it generates a message such as "Please separate PET bottles as plastic waste."

[0072] Step 7:

[0073] The server constructs a response in JSON format that includes the generated sorting method and sends it to the terminal.

[0074] Step 8:

[0075] The terminal receives and analyzes the response from the server.

[0076] Step 9:

[0077] The terminal displays the analyzed sorting method to the user.

[0078] Step 10:

[0079] If the user has any further questions about the sorting method, they can use the chat function on the device to type in their questions.

[0080] Step 11:

[0081] The terminal sends the user's question to the server.

[0082] Step 12:

[0083] The server receives the question and uses natural language processing (NLP) algorithms to analyze the question and generate an answer.

[0084] Step 13:

[0085] The server sends the generated response to the terminal.

[0086] Step 14:

[0087] The terminal receives the response and displays it to the user.

[0088] Step 15:

[0089] A user selects environmental education content (e.g., a video about recycling) on ​​the device to watch.

[0090] Step 16:

[0091] The terminal notifies the server of the user's viewing actions.

[0092] Step 17:

[0093] When the user finishes watching the video, the device sends that information to the server.

[0094] Step 18:

[0095] The server confirms that the viewing has been completed and awards points to the user.

[0096] Step 19:

[0097] The server transmits the awarded point information to the terminal.

[0098] Step 20:

[0099] The terminal displays the updated points information to the user.

[0100] Step 21:

[0101] When a user participates in a discussion or event in an online community, the terminal notifies the server of the participation information.

[0102] Step 22:

[0103] The server records participation information and awards rewards and badges to users as appropriate.

[0104] Step 23:

[0105] The server sends reward and badge information to the terminal.

[0106] Step 24:

[0107] The terminal displays reward and badge information to the user.

[0108] Example 1

[0109] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0110] In conventional waste sorting systems, users have difficulty accurately identifying the type of waste and rarely have detailed knowledge of how to sort it. Furthermore, existing systems make it difficult for users to receive ongoing environmental education or community activity support, resulting in insufficient improvement in users' environmental awareness. Therefore, there is a need for a system that promotes sustainable lifestyles by supporting proper waste sorting and providing users with ongoing environmental education and community activities.

[0111] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0112] In this invention, the server includes: a means for a user to acquire an image of waste; a means for transmitting image data from a terminal to the server; a means for the server to analyze the received image data and identify the type of waste; a means for generating a recommended sorting method based on the type of waste identified by the server; a means for transmitting the generated sorting method to the terminal and displaying it; a means for the user to input additional questions and transmit them from the terminal to the server; a means for the server to analyze the questions and generate answers using natural language processing; and a means for transmitting the generated answers to the terminal and displaying them. This allows users to easily and accurately sort waste and obtain necessary information in real time. Furthermore, the system includes a means for transmitting information about viewing environmental education content to the server and awarding points, and a means for recording participation information in the online community and awarding rewards and badges, thereby improving users' environmental awareness and supporting sustainable lifestyles.

[0113] "User" refers to an individual who uses the system to perform operations related to waste separation and environmental education.

[0114] A "terminal" is a communication device used by a user, and specifically includes a smartphone, tablet, computer, etc.

[0115] A "server" is a computing device that receives and processes data sent from a terminal, and is responsible for analyzing the data and generating results.

[0116] "Image data" refers to photographic information of waste taken by a user using a terminal.

[0117] "Image analysis" refers to the computational process in which the server processes image data to determine the type of waste.

[0118] "Sorting methods" refer to specific procedures and rules for properly classifying and disposing of waste.

[0119] "Natural language processing" refers to a technology that allows a server to understand questions or messages entered by users and generate appropriate answers.

[0120] "Environmental education content" refers to teaching materials and resources that users can view to increase their environmental knowledge and awareness.

[0121] "Points" refer to rewards given to users for viewing environmental education content or using the system.

[0122] "Online community" refers to a network group in which users can participate to exchange information and engage in activities related to environmental protection.

[0123] "Rewards" refers to incentives such as badges and points that users receive through their activities in online communities.

[0124] "Badge" refers to a digital award or recognition given to users to recognize their achievements in online communities.

[0125] The present invention is a system that supports users in separating waste materials easily and appropriately. This system is composed of elements such as terminals, servers, and users. Specifically, it operates as follows.

[0126] Taking photos of trash and sending the data

[0127] Users take photos of waste using a device (e.g., a smartphone or tablet). The captured photo data is sent from the device to a server. This transmission is performed using a REST API using the HTTP protocol.

[0128] Image analysis on the server

[0129] The server passes the received image data to an image analysis algorithm. Specific algorithms used include image classification models such as Tensorflow (registered trademark) and OpenCV. The image analysis algorithm identifies the shape and characteristics of the waste and identifies its type (e.g., plastic bottles, paper, metal, etc.).

[0130] Presentation of sorting methods

[0131] The server generates an appropriate sorting method based on the results of image analysis. For example, if it identifies a plastic bottle, it generates a message saying, "Please separate plastic bottles as plastic waste." This message is sent from the server to the terminal and displayed to the user.

[0132] Real-time chat support

[0133] If the user has any additional questions about the sorting method, they can enter them using the chat function on their device. These questions are then sent from the device to the server. The server then analyzes the received questions using a natural language processing algorithm (e.g., GPT-3 (registered trademark)) and generates an appropriate answer. The generated answer is then sent to the device and displayed to the user.

[0134] Environmental education and point redemption

[0135] The user views environmental education content (for example, videos or articles about recycling) on ​​the device. The viewing information is sent from the device to the server, which then confirms that the viewing is complete and awards points to the user. For example, a message saying "10 points awarded" is displayed on the device.

[0136] Community participation and support

[0137] Users join online communities and participate in discussions and events related to environmental protection. Participation information is sent from the device to the server, which records the participation information and awards rewards and badges as necessary. For example, a message saying "You have earned the Recycling Master badge" is displayed on the device.

[0138] Specific examples

[0139] Example 1: Taking photos of garbage and showing how to separate it

[0140] A user takes a photo of a plastic bottle with their device and sends the data to the server. The server recognizes it as a plastic bottle, generates a message saying "Please separate plastic bottles as plastic waste," and sends it to the device. The message is displayed on the user's device, allowing the user to separate the bottles correctly.

[0141] Example of a generative AI model input prompt: "I took a photo of some waste. How should I sort it?"

[0142] Example 2: Watching environmental education videos and earning points

[0143] A user watches a video about recycling on their device, and after completing the viewing, the information is sent to the server. The server confirms the completion of the viewing and awards the user 10 points. The user's device displays "10 points awarded," encouraging environmental education.

[0144] Example of a generative AI model input prompt: "I watched the recycling video to the end. Please give me points."

[0145] By combining these functions, the system helps users separate waste accurately and efficiently, and increases environmental awareness. It supports the development of sustainable lifestyles through real-time advice, reward points, and community participation.

[0146] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0147] Step 1: User takes a photo on their device

[0148] Users take photos of waste using devices such as smartphones or tablets. The input for this process is the image of waste taken by the user using the device's camera function, and the output is image data. Specifically, the user launches the camera app, points it at the waste, and presses the shutter button.

[0149] Step 2: The device sends the photo data to the server

[0150] The device sends the captured photo data to the server. The input to this process is the image data stored on the device, and the output is the transfer of the image data to the server. A REST API using the HTTP protocol is used to send the data. Specifically, the application on the device generates an HTTP request containing the image data and sends it to the specified server endpoint.

[0151] Step 3: The server receives the image data.

[0152] The server receives image data sent from the terminal. The input of this process is image data sent via the network, and the output is image data saved on the server. Specifically, the server receives an HTTP request and temporarily saves the image data.

[0153] Step 4: The server performs image analysis

[0154] The server passes the image data to an image analysis algorithm. The input for this process is the image data stored on the server, and the output is an analysis result indicating the type of waste. Specifically, the image data is analyzed using an image classification model using TensorFlow and OpenCV to obtain classification results such as PET bottles, paper, and metal.

[0155] Step 5: The server generates the sorting method

[0156] The server generates a sorting method based on the results of image analysis. The input to this process is the result of image analysis, and the output is a message indicating the sorting method. Specifically, it generates a message such as "Please separate PET bottles as plastic waste" based on a predefined database and rules.

[0157] Step 6: The server sends a classification method message to the terminal.

[0158] The server sends the generated sorting method message to the terminal. The input of this process is the generated sorting method message, and the output is message delivery to the terminal. Specifically, the server sends the message to the terminal using an HTTP response or a messaging queue.

[0159] Step 7: The device displays a sorting method message to the user.

[0160] The terminal displays the received message on the sorting method to the user. The input of this process is the message on the sorting method sent from the server, and the output is the message displayed on the terminal screen. Specifically, the terminal application receives the message and displays it on the user interface (UI).

[0161] Step 8: User enters follow-up question into terminal

[0162] The user enters additional questions about the sorting method using the chat function on the device. The input for this process is the user's question text, and the output is the question data entered into the device. Specifically, the user opens a chat app and enters, "How should I dispose of this bottle cap?"

[0163] Step 9: The device sends the query data to the server

[0164] The terminal sends the entered question data to the server. The input of this process is the question data entered into the terminal, and the output is the transfer of the question data to the server. Specifically, the terminal generates an HTTP request including the question data and sends it to the server.

[0165] Step 10: The server performs natural language processing and generates an answer

[0166] The server analyzes the received question data using a natural language processing algorithm and generates an answer. The input for this process is the question data, and the output is the generated answer text. Specifically, it uses a natural language processing model such as GPT-3 to generate the answer "Please separate the caps as burnable garbage."

[0167] Step 11: The server sends the response data to the terminal.

[0168] The server sends the generated answer data to the terminal. The input of this process is the generated answer text, and the output is the transfer of the answer data to the terminal. Specifically, the server generates an HTTP response including the answer data and sends it to the terminal.

[0169] Step 12: The device displays the response data to the user

[0170] The terminal displays the received answer data to the user. The input of this process is the answer text sent from the server, and the output is the answer displayed on the terminal screen. Specifically, the terminal application receives the answer data and displays it on the user interface (UI).

[0171] In this way, the system helps users separate waste accurately and easily, provides additional information in real time, and provides a mechanism to increase users' environmental awareness through environmental education and community activities.

[0172] (Application example 1)

[0173] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0174] In recent years, the increase in the use of food delivery services has led to a significant increase in waste, including containers and packaging materials. While this has created a demand for proper waste separation, users find it difficult to find the time to do so due to their busy daily lives. Furthermore, the complicated separation method often leads to incorrect separation. Furthermore, the lack of incentives for separating waste often leads to a decline in motivation. A system is needed to solve these problems, encourage users to separate waste effectively, and raise their environmental awareness.

[0175] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0176] In this invention, the server includes means for a user to take a photo of waste, means for transmitting the photo data to the server, means for the server to analyze the photo data and determine the type of waste, means for generating a recommended sorting method based on the type of waste determined by the server, means for transmitting the sorting method generated by the server to the user's terminal and displaying it, means for the user to be awarded points according to their waste sorting and displaying the point information, and means for the user to view environmental education content and transmit the viewed information to the server. This allows users to easily learn the appropriate sorting method for food delivery waste, increasing their incentive to sort waste and improving overall environmental awareness.

[0177] "User" refers to a person who uses the system to separate waste.

[0178] "Waste" refers to unwanted items and garbage.

[0179] "Photo" refers to image data that a user takes to record the state of waste.

[0180] "Means" refers to the methods or devices that a system uses to achieve its goals.

[0181] "Server" refers to a central management device that receives and analyzes data sent by users, and generates and transmits the necessary information.

[0182] "Data" refers collectively to information processed within the system, such as photos taken by users and information generated by the system.

[0183] "Analysis" refers to the process by which the server processes the received photo data and identifies the type of waste.

[0184] "Sorting methods" refers to instructions and guidelines for properly disposing of and classifying waste.

[0185] "Terminal" refers to the device (such as a smartphone or tablet) that a user uses to access the system.

[0186] "Points" refer to rewards given to users for properly sorting waste and viewing environmental education content.

[0187] "Point information" refers to information indicating the status and history of points given to a user.

[0188] "Environmental education content" refers to educational materials such as videos and articles that help users increase their environmental awareness through learning.

[0189] This invention is a system that supports the separation of waste generated by food delivery. The system allows users to take photos of waste and send them to a server, which then presents the user with appropriate separation methods and provides a function that awards points for following the separation methods. The following describes in detail an embodiment of this invention.

[0190] 1. Hardware and Software Requirements

[0191] Hardware: Smartphones, servers

[0192] Software: OpenCV, Scikit-learn, Joblib, Natural Language Processing algorithms

[0193] 2. System Configuration

[0194] User's device: The user takes a photo of the waste using a smartphone or tablet, and the device sends the photo data to the server via the Internet.

[0195] Server: The server passes the received photo data to an image analysis algorithm to identify the type of waste. Based on the analysis results, it generates a recommended sorting method and sends it to the user's device. It also awards points based on the user's behavior and manages the points history.

[0196] Image analysis algorithm: Images are preprocessed using OpenCV, and waste type is identified using a Scikit-learn-based model that has been trained on image data of various waste materials.

[0197] Natural language processing algorithms: When users enter a question, the system analyzes the question and generates an appropriate answer, allowing users to receive real-time support on how to sort in detail.

[0198] 3. System Operation

[0199] Users take photos of waste generated during food delivery and send them to a server via the app. The server then passes the received photos through an image analysis algorithm to determine the type of waste. For example, if the waste is a plastic container, a message saying "Please separate it as plastic waste" is generated and displayed on the user's device. When users follow the sorting instructions, they are awarded points based on their actions, which can be used as incentives such as discounts on delivery.

[0200] Specific examples

[0201] 1. Taking photos of trash and showing how to separate it

[0202] The user takes a photo of the food container after delivery with their smartphone and sends it to the server via the app. The server analyzes the photo and displays a message to the user saying, "Please separate this plastic container as burnable garbage."

[0203] 2. Points Award

[0204] When users correctly separate their trash according to the provided separation method, the app's internal history is updated and they are awarded recycling points, which can be used for discounts on their next food delivery.

[0205] Prompt Sentence Examples

[0206] User Question: "Which garbage bag should I put this plastic container in?"

[0207] Prompt for generative AI model: "Please tell me the proper way to separate plastic containers."

[0208] User Question: "How do I dispose of paper bags?"

[0209] Prompt for generative AI model: "How do I dispose of paper bags?"

[0210] In this way, the present invention becomes a powerful tool to help users properly separate waste and promote environmental awareness.

[0211] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0212] Step 1:

[0213] The user takes a photo of the waste used in food delivery using their smartphone. To do this, the user launches the application and takes a photo using the camera function. The input is the photo of the waste taken. The output is that this photo data is saved on the device.

[0214] Step 2:

[0215] The device initiates network communication to send the captured photo data to the server. The input is the photo data, and the output is the photo data sent to the server. A secure communication method such as HTTPS is used to send the data.

[0216] Step 3:

[0217] The server passes the received photo data to the image analysis algorithm, which uses OpenCV to pre-process the photos. This pre-processing involves resizing and normalizing the photos to convert the image data into an analyzable format. The input is the photo data sent to the server, and the output is the pre-processed image data.

[0218] Step 4:

[0219] The server passes the preprocessed image data to a waste classification model. Here, a pre-trained model from Scikit-learn is used. The model analyzes the image data and identifies the type of waste. The input is the preprocessed image data, and the output is the analysis result. For example, it may be identified as a plastic container.

[0220] Step 5:

[0221] The server generates a recommended sorting method based on the analysis results. The input is the type of waste (analysis result), and the output is a message about the sorting method. For example, a message such as "Please sort this plastic container as plastic waste" is generated.

[0222] Step 6:

[0223] The server sends the generated message of the sorting method to the terminal. The input is the generated message of the sorting method, and the output is the message sent to the terminal. The message instructs the user on the sorting method.

[0224] Step 7:

[0225] The terminal displays the message of the sorting method received from the server to the user. The user checks the displayed message and understands the sorting method. The input is the message received from the server, and the output is the message displayed to the user.

[0226] Step 8:

[0227] The user must confirm the operation before points can be awarded for sorting waste. After the user completes the sorting task, the user sends the information to the server via their device. The input is the user's sorting task completion information, and the output is the information sent to the server.

[0228] Step 9:

[0229] The server receives the user's sorting completion information and awards points. The input is the user's sorting completion information, and the output is the points awarded to the user. The points can be used as a discount for the next delivery.

[0230] Step 10:

[0231] The server receives information about the user's browsing of environmental education content and awards points based on that information. The input is the user's browsing information of environmental education content, and the output is the point information awarded to the user.

[0232] Step 11:

[0233] When a user uses a discount or benefit, the point information is checked on the terminal. The terminal displays the user's point balance and history. The input is the user's point information, and the output is the point information displayed on the terminal.

[0234] Through the above steps, a system is realized in which users can properly separate their waste and earn points to receive a discount on their next food delivery.

[0235] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0236] This invention is a system that supports users in taking photos of waste and sorting it appropriately. This system consists of components: a terminal, a server, a user, and an emotion engine.

[0237] System Overview

[0238] 1. Taking photos of trash and sending the data

[0239] Users take photos of waste using their devices (smartphones or tablets), and the devices send the photo data to the server. For example, if a user takes a photo of a plastic bottle, the photo data is sent to the server.

[0240] 2. Image analysis on the server

[0241] The server passes the received photo data to an image analysis algorithm, which determines the type of waste (e.g., plastic bottle, paper, metal, etc.) based on its shape and characteristics. After analysis, the server confirms that the waste is a plastic bottle.

[0242] 3. Presentation of sorting methods

[0243] Based on the analysis results, the server generates a sorting method appropriate for the PET bottles. For example, it generates a message such as "Please separate PET bottles as plastic waste." This message is sent to the terminal and displayed to the user.

[0244] 4. Real-time chat support

[0245] If the user has any further questions about the sorting method, they can use the chat function on their device to input their question. For example, they can input, "How should I dispose of this bottle cap?" The device sends this question to the server, which then analyzes the question using a natural language processing algorithm and generates an answer. The answer generated, "Please sort the cap as burnable waste," is sent to the device and displayed to the user.

[0246] 5. Environmental education and point redemption

[0247] The user views environmental education content (e.g., videos or articles about recycling). The device sends the viewing information to the server. The server confirms that the viewing has been completed and awards points to the user based on that information. The point information is updated in real time on the user's device and displayed. For example, it may say, "10 points awarded."

[0248] 6. Community Participation and Support

[0249] Users join online communities and participate in discussions and events related to environmental protection. The device notifies the server of their participation. The server records the participation information and awards rewards and badges as necessary. This information is also sent to the device and displayed to the user. For example, the server may notify the user that they have earned a recycling master badge.

[0250] 7. Improving user experience by incorporating an emotion engine

[0251] The emotion engine has the ability to analyze a user's facial expressions and voice and recognize their emotions. For example, when a user takes a photo or types a question using the chat function, the facial expression and voice data are acquired. The device sends this data to the server, and the server uses the emotion engine to recognize the user's emotions.

[0252] The recognized emotion information is taken into consideration when presenting methods and answers. For example, if the user is confused, a more detailed and easy-to-understand explanation is presented. If the user is satisfied, a simple message is displayed.

[0253] Specific examples

[0254] Example 1: Taking photos of garbage and showing how to separate it

[0255] The user takes a photo of a plastic bottle with their device and sends the data to the server. The server recognizes it as a plastic bottle and generates a message saying "Please separate plastic bottles as plastic waste" and sends it to the device. The message is displayed on the user's device, allowing the user to separate the bottles correctly. The user's facial expression at this time is also recorded and sent as data to the server.

[0256] Example 2: Watching environmental education videos and earning points

[0257] The user watches a video about recycling on their device, and after watching, the information is sent to the server. The server confirms that the user has watched the video and awards the user 10 points. The user's device displays a message saying "10 points awarded," encouraging them to become more environmentally educated. The user's emotions are also recorded, and if the user's satisfaction after watching is high, this information is reflected in the next content suggestions.

[0258] This invention is a support system that helps users easily separate their garbage and raise environmental awareness. It is a powerful tool for achieving sustainable living by providing real-time advice, reward points, and encouraging community participation. Furthermore, the incorporation of an emotion engine improves the user experience and provides more effective education and support.

[0259] The processing flow will be explained below.

[0260] Step 1:

[0261] The user launches the camera app on the device and takes a photo of the waste (e.g., a plastic bottle).

[0262] Step 2:

[0263] The device temporarily stores the captured photo data, acquires the user's facial expression data, and prepares an HTTP POST request to send to the server.

[0264] Step 3:

[0265] The device sends the photo data and facial expression data to the server via an HTTP POST request.

[0266] Step 4:

[0267] The server receives the HTTP POST request and retrieves the photo data and facial expression data.

[0268] Step 5:

[0269] The server uses an AI model for image analysis to analyze the photo data and determine the type of waste, using feature extraction and identification algorithms.

[0270] Step 6:

[0271] The server uses an emotion engine to analyze the facial expression data and recognize the user's emotional state (e.g., joy, confusion, indifference, etc.).

[0272] Step 7:

[0273] Based on the analysis results, the server generates an appropriate method for separating waste according to the type of waste. The content and tone of the message are adjusted depending on the user's emotional state. For example, for a confused user, the server generates a detailed explanation such as "Please separate PET bottles as plastic waste. Please see below for detailed instructions."

[0274] Step 8:

[0275] The server constructs a response in JSON format that includes the generated sorting method and sends it to the terminal.

[0276] Step 9:

[0277] The terminal receives and analyzes the response from the server.

[0278] Step 10:

[0279] The device displays the analyzed classification method to the user, and also displays additional support messages depending on the user's emotional state.

[0280] Step 11:

[0281] If the user has any further questions about the sorting method, they can use the chat function on the device to type in their questions.

[0282] Step 12:

[0283] The terminal transmits the user's question together with the reacquired facial expression data to the server.

[0284] Step 13:

[0285] The server receives the question and facial expression data, analyzes the question using natural language processing (NLP) algorithms, and reconfirms the user's emotional state.

[0286] Step 14:

[0287] The server generates an appropriate response, again adjusting it based on the emotional state, for example adding a follow-up message such as "Please separate the caps as burnable trash. If you have any further questions, please let us know."

[0288] Step 15:

[0289] The server sends the generated response to the terminal.

[0290] Step 16:

[0291] The terminal receives the response and displays it to the user.

[0292] Step 17:

[0293] A user selects environmental education content (e.g., a video about recycling) on ​​the device to watch.

[0294] Step 18:

[0295] The terminal notifies the server of the user's selection and facial expression data.

[0296] Step 19:

[0297] When the user finishes watching the video, the terminal transmits viewing completion information and facial expression data during viewing to the server.

[0298] Step 20:

[0299] The server confirms that the viewing has been completed and awards points to the user.

[0300] Step 21:

[0301] The server takes into consideration the points awarded and the user's emotional state while watching, adjusts the next recommended content, and sends it to the terminal.

[0302] Step 22:

[0303] The device displays updated point information and recommended content tailored to the user's emotional state.

[0304] Step 23:

[0305] When a user participates in a discussion or event in an online community, the terminal notifies the server of the participation information.

[0306] Step 24:

[0307] The server records participation information and awards rewards and badges to users as appropriate.

[0308] Step 25:

[0309] The server sends reward and badge information to the terminal.

[0310] Step 26:

[0311] The device displays reward and badge information to the user, and also collects emotional data from the user during the event to reflect in the next event information.

[0312] Example 2

[0313] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0314] Currently, waste sorting is generally done manually, making it difficult for users to consistently and accurately understand and follow the sorting method. Furthermore, when users have questions about the sorting method, they often cannot immediately resolve them, preventing them from sorting accurately. Furthermore, there is a lack of support to maintain motivation for waste sorting and flexible support based on users' emotions. There are also limited incentives for viewing environmental education content, making it necessary to improve eco-consciousness.

[0315] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0316] In this invention, the server includes: a device that transmits photo data and additional data to the server; a server that passes the photo data received from the device to an image analysis algorithm to identify the type of waste; a device that generates a recommended sorting method based on the type of waste identified by the server; a device that transmits the generated sorting method to the terminal and displays it to the user; a device that acquires the user's facial expression and voice data and transmits it to the server; and a server that recognizes the user's emotions using an emotion analysis algorithm and uses that information to present the sorting method. This allows users to easily and accurately sort waste and improves the user experience by providing real-time question and answer services and flexible support based on emotions. Furthermore, a points system for viewing environmental education content can improve eco-consciousness and promote sustainable behavior.

[0317] "User" refers to an individual or organization that uses this system to separate waste.

[0318] "Terminal" refers to an electronic device such as a smartphone or tablet used by a user.

[0319] A "server" refers to a computer system that receives and analyzes data sent from a terminal and sends the results back to the terminal.

[0320] "Photo data" refers to image data of waste photographed by a user using a terminal.

[0321] "Additional data" refers to additional information such as GPS data and timestamps that are acquired along with the photo data.

[0322] "Image analysis algorithm" refers to a machine learning algorithm that analyzes received photo data and determines the type of waste.

[0323] "Waste type" refers to the category into which waste is classified, examples include plastic bottles, paper, metal, etc.

[0324] "Sorting method" refers to the appropriate disposal method recommended to the user based on the identified type of waste.

[0325] "Facial expression data" refers to data relating to the user's facial expressions.

[0326] "Voice data" refers to voice data relating to the content of a user's speech.

[0327] An "emotion analysis algorithm" refers to an algorithm that analyzes facial expression data and voice data to estimate a user's emotions.

[0328] A "natural language processing algorithm" refers to an algorithm that analyzes a user's question or input text and generates an appropriate answer.

[0329] "Environmental education content" refers to information such as videos and articles that are intended to educate people about recycling and environmental protection.

[0330] "Points" refer to rewards given to users for viewing environmental education content and engaging in proper waste sorting activities.

[0331] The system of the present invention assists users in taking photos of waste and sorting it appropriately. This system is implemented by users using devices such as smartphones or tablets, which communicate with a server. The system also incorporates an emotion engine and natural language processing algorithms to improve the user experience.

[0332] Hardware and Software

[0333] 1. Terminal

[0334] Electronic devices such as smartphones and tablets.

[0335] Includes a camera, GPS module, microphone, etc.

[0336] 2. Server

[0337] A computer system that processes and analyzes data.

[0338] Implemented database, image analysis algorithm, natural language processing algorithm, and sentiment analysis algorithm.

[0339] 3. Software

[0340] Image analysis algorithms (e.g. TensorFlow)

[0341] Natural language processing algorithms (e.g., GPT-3 by OpenAI (registered trademark))

[0342] Sentiment analysis algorithms (e.g., Microsoft® Azure® Cognitive Services)

[0343] Data processing and calculation

[0344] 1. Taking photos of trash and sending the data

[0345] The user uses the terminal to take a photo of the waste.

[0346] The device sends the photo data along with GPS data and a timestamp to the server.

[0347] 2. Image analysis on the server

[0348] The server passes the received photo data to an image analysis algorithm.

[0349] The algorithm identifies the type of waste (plastic bottles, paper, metal, etc.).

[0350] 3. Presentation of sorting methods

[0351] The server generates an appropriate sorting method based on the results of image analysis.

[0352] The server sends the generated sorting method to the terminal and displays it to the user.

[0353] 4. Real-time chat support

[0354] The user enters a question using the chat function of the device.

[0355] The server uses natural language processing algorithms to analyze the question and generate an answer.

[0356] The generated answer is sent to the terminal and displayed to the user.

[0357] 5. Environmental education and point redemption

[0358] The user views the environmental education content on the device.

[0359] The terminal transmits the browsing information to the server.

[0360] The server confirms that the viewing is complete and gives points to the user.

[0361] Points information is displayed on the device in real time.

[0362] 6. Community Participation and Support

[0363] Users participate in online communities and engage in activities.

[0364] The terminal transmits the participation information to the server.

[0365] The server will award rewards and badges based on activity records.

[0366] Reward information will be displayed on the device.

[0367] 7. Use of Emotion Engines

[0368] When the device takes a photo or uses the chat function, it collects the user's facial expressions and voice data.

[0369] The terminal transmits this data to the server.

[0370] The server analyzes the user's emotions using an emotion analysis algorithm.

[0371] Based on the analysis results, sorting methods and responses are individually adjusted to improve the user experience.

[0372] Specific examples

[0373] Example 1: Taking photos of garbage and showing how to separate it

[0374] The user takes a photo of the plastic bottle with the device and sends the data to the server.

[0375] The server recognizes it as a plastic bottle, generates a message saying "Please separate plastic bottles as plastic waste," and sends it to the terminal.

[0376] The message will be displayed on the user's terminal, allowing the user to properly distinguish between the two.

[0377] The user's facial expression at this time is also recorded and sent as data to the server.

[0378] Example prompt sentence:

[0379] Photo of a plastic bottle. How should I dispose of it?

[0380] Example 2: Watching environmental education videos and earning points

[0381] Users watch videos about recycling on their devices, and after they finish watching, the information is sent to the server.

[0382] The server confirms that the viewing has been completed and awards the user 10 points.

[0383] The user's device will display the message "10 points awarded," encouraging environmental education.

[0384] The user's emotions are also recorded at this time, and if the user is highly satisfied after viewing, this information is reflected in the next content suggestions.

[0385] Example prompt sentence:

[0386] I have completed watching the recycling video. How many points did I earn?

[0387] This invention is a support system that helps users easily separate their garbage and raise environmental awareness. It is a powerful tool for achieving sustainable living by providing real-time advice, reward points, and encouraging community participation. Furthermore, the incorporation of an emotion engine improves the user experience and provides more effective education and support.

[0388] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0389] Step 1:

[0390] Garbage photography

[0391] Operation:

[0392] Users take photos of waste using the camera function on their device, such as a smartphone or tablet.

[0393] input:

[0394] The physical object of waste (e.g., plastic bottles).

[0395] output:

[0396] Save as image data on the device.

[0397] Specific operation:

[0398] The user opens the camera app, points the lens at the waste, and presses the shutter button. The device takes a photo and saves the image data to the internal storage. GPS data and a timestamp are also acquired.

[0399] Step 2:

[0400] Sending photo data

[0401] Operation:

[0402] The device sends the photo data and additional data (GPS data, timestamp) to the server.

[0403] input:

[0404] Image data, GPS data, timestamp.

[0405] output:

[0406] Data transmission to the server completed.

[0407] Specific operation:

[0408] The user presses the "Send" button in the application on the device. The device uploads the photo data and additional data to the server via the Internet. The server receives the data and temporarily stores it in a database.

[0409] Step 3:

[0410] Receiving and storing image data

[0411] Operation:

[0412] The server receives the photo data and additional data sent from the terminal and temporarily stores them.

[0413] input:

[0414] Image data, GPS data, timestamp.

[0415] output:

[0416] Storing data in a database.

[0417] Specific operation:

[0418] When the server receives the data sent from the terminal, it temporarily stores the image data and additional data in the database, and creates a new entry in the database according to the existing data structure.

[0419] Step 4:

[0420] Starting image analysis

[0421] Operation:

[0422] The server passes the received photo data to an image analysis algorithm to determine the type of waste.

[0423] input:

[0424] Image data.

[0425] output:

[0426] Analysis results (type of waste).

[0427] Specific operation:

[0428] The server invokes an image analysis algorithm (e.g., TensorFlow) to analyze the stored image data. The algorithm extracts features and identifies the type of waste in the image. The results are returned as text, such as "PET bottle."

[0429] Step 5:

[0430] Generation of separation methods

[0431] Operation:

[0432] The server generates an appropriate classification method based on the analysis results.

[0433] input:

[0434] Analysis results (type of waste).

[0435] output:

[0436] Message on how to sort waste.

[0437] Specific operation:

[0438] The server selects a message template based on the type of waste and generates specific sorting instructions, such as "Please separate PET bottles as plastic waste." The generated message is temporarily stored.

[0439] Step 6:

[0440] Sending sorting instructions

[0441] Operation:

[0442] The server transmits the generated sorting method to the terminal.

[0443] input:

[0444] Message on how to sort waste.

[0445] output:

[0446] Message sent to device completed.

[0447] Specific operation:

[0448] The server sends a message including the classification method to the terminal. This message reaches the terminal via the Internet. The terminal notifies the user of the received message.

[0449] Step 7:

[0450] Display of sorting method

[0451] Operation:

[0452] The terminal displays the received sorting method to the user.

[0453] input:

[0454] Message on how to sort waste.

[0455] output:

[0456] Displayed on the user screen.

[0457] Specific operation:

[0458] The device will then display the received sorting instructions in a pop-up notification or in a message box within the app, showing the user the message, "Please separate PET bottles as plastic waste."

[0459] Step 8:

[0460] Entering user questions

[0461] Operation:

[0462] Users can use the chat function on their device to enter questions about sorting methods.

[0463] input:

[0464] User text input (question).

[0465] output:

[0466] Prepare to send question data from the terminal to the server.

[0467] Specific operation:

[0468] A user opens a chat application and types a question such as, "How should I dispose of this bottle cap?" Once the question is typed, the device prepares the data to be sent to the server.

[0469] Step 9:

[0470] Submitting Question Data

[0471] Operation:

[0472] The terminal transmits the entered question to the server.

[0473] input:

[0474] User text input (question).

[0475] output:

[0476] Question data has been sent to the server.

[0477] Specific operation:

[0478] The device sends the question data entered by the user to the server via the Internet. After sending, a message saying "Question has been sent" is displayed on the screen.

[0479] Step 10:

[0480] Question analysis and answer generation

[0481] Operation:

[0482] The server passes the question data to a natural language processing algorithm to generate an answer.

[0483] input:

[0484] User text input (question).

[0485] output:

[0486] Analysis result (answer).

[0487] Specific operation:

[0488] The server analyzes the question using a natural language processing algorithm (e.g., GPT-3). The algorithm understands the context and generates a specific answer, such as "Please separate the caps as burnable garbage." The generated answer is temporarily stored.

[0489] Step 11:

[0490] Submitting and viewing answers

[0491] Operation:

[0492] The server sends the generated answer to the terminal, which displays it to the user.

[0493] input:

[0494] Analysis result (answer).

[0495] output:

[0496] The response is sent to the device and displayed on the user's screen.

[0497] Specific operation:

[0498] The server sends a message containing the answer to the terminal, which displays the received answer in real time, showing the user the message "Please separate the caps as burnable garbage."

[0499] Step 12:

[0500] Viewing environmental education content and earning points

[0501] Operation:

[0502] The user watches the environmental education content, and the terminal sends the viewing completion information to the server. The server confirms the viewing completion and awards points.

[0503] input:

[0504] Viewing data for educational content.

[0505] output:

[0506] Points are awarded.

[0507] Specific operation:

[0508] The user watches an environmental education video, and after watching it, the device sends the information to the server. The server checks the viewing data and awards the user 10 points. The device then displays a notification to the user that points have been awarded.

[0509] Step 13:

[0510] Community Participation and Rewards

[0511] Operation:

[0512] When a user joins an online community, the terminal sends the participation information to the server, which records the participation information and grants rewards.

[0513] input:

[0514] Community participation information.

[0515] output:

[0516] Rewarding.

[0517] Specific operation:

[0518] Users join online communities and participate in discussions and events. The device sends this information to the server, which records the participation information and awards "recycling master badges" as necessary. The device then notifies the user of rewards.

[0519] Step 14:

[0520] Acquiring and analyzing emotion data

[0521] Operation:

[0522] The device captures the user's facial expressions and voice data and sends it to the server, which then analyzes their emotions using an emotion analysis algorithm.

[0523] input:

[0524] Facial expression data, voice data.

[0525] output:

[0526] Emotion analysis results.

[0527] Specific operation:

[0528] When a user takes a photo or posts a question via chat, the device uses a camera and microphone to capture facial and voice data. The device then sends this data to a server, which then runs an emotion analysis algorithm to identify the user's emotions. The analysis results are then reflected in the next step.

[0529] Step 15:

[0530] Regulating responses based on emotions

[0531] Operation:

[0532] Based on the results of the emotion analysis, the server individually adjusts the classification method and responses and sends them to the device.

[0533] input:

[0534] Emotion analysis results.

[0535] output:

[0536] Tailored response.

[0537] Specific operation:

[0538] The server generates a detailed and easy-to-understand explanation if the user is confused, or a concise message if the user is satisfied. The tailored response is sent to the terminal and displayed to the user.

[0539] (Application example 2)

[0540] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0541] Conventional waste sorting support systems help users sort waste accurately, but they lack the functionality to provide direct incentives for that behavior. Furthermore, there is no mechanism to further improve the accuracy of waste sorting and encourage users to continue using the system. As a result, users' interest and motivation declined, and participation in sustainable environmental protection activities did not expand as expected. Furthermore, because there was no point system linked to an electronic payment service, the benefits to users could not be directly linked to their daily lives.

[0542] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for a user to take an image of waste, means for transmitting image data to the server, means for the server to analyze the image data and determine the type of waste, means for generating a recommended sorting method based on the type of waste determined by the server, means for transmitting the sorting method generated by the server to the user's terminal and displaying it, and means for awarding points according to waste sorting and making the points usable for electronic payment services. This allows users to receive direct incentives by properly sorting waste, promoting continued use and expanding participation in sustainable environmental protection activities.

[0543] A "user's device" refers to an electronic device operated by a user, and mainly refers to a smartphone, tablet, or PC.

[0544] "Image data" refers to photographic data of waste that a user takes with a terminal and sends to a server.

[0545] "Server" refers to a computer system for receiving and analyzing image data via a network.

[0546] "Waste type" refers to the classification of what material or type the waste falls into (e.g., plastic bottles, paper, metal, etc.).

[0547] "Sorting method" refers to the appropriate disposal method or treatment procedure recommended based on the type of waste.

[0548] "Points" are incentives given to users who properly separate their waste, and refer to virtual currency or electronic money that can be used in electronic payment services.

[0549] "Electronic payment service" refers to an online payment system that enables users to pay for goods and services via the Internet.

[0550] "Environmental education content" refers to media such as videos, articles, and quizzes that provide users with knowledge and information about environmental protection and recycling.

[0551] "Natural language processing" refers to computer algorithms and technologies that analyze text data entered by users, understand its meaning, and generate appropriate answers.

[0552] This system allows users to take pictures of waste and properly separate it to earn points, which can then be used for electronic payment services. The system mainly consists of a user terminal, a server, an image analysis algorithm, a natural language processing algorithm, and an electronic payment platform.

[0553] 1. Use of User Devices

[0554] Users take photos of waste using devices such as smartphones or tablets. This photo data is sent to a server via the Internet. The user's device must have a camera function and be connected to the Internet.

[0555] 2. Data reception and analysis by the server

[0556] The server receives image data sent from the user's device and uses image analysis algorithms such as TensorFlow to determine the type of waste in the received image data. Depending on the type of waste, the server generates a recommended sorting method and sends it to the user's device.

[0557] 3. Display of recommended sorting methods

[0558] The user terminal receives the sorting instructions sent from the server and displays them. For example, a message such as "Please separate PET bottles as plastic waste" is displayed.

[0559] 4. Points allocation and real-time display

[0560] If the sorting is done properly, the server will award points to the user. This point information is updated in real time on the user's device, and a message such as "5 points awarded" is displayed. These points can be used for actual payments by linking with an electronic payment service.

[0561] 5. Environmental education content and point linkage

[0562] Users can also view environmental education content, including videos and articles about recycling. The server assigns points based on the viewing information and sends the information to the user's device. For example, the server displays, "You have watched a video about recycling and received 10 points."

[0563] 6. Question Answering Using Natural Language Processing

[0564] When a user enters a question about sorting, the question is sent to the server. The server analyzes the question using a natural language processing algorithm, generates an appropriate answer, and sends it to the user's device. The answer displayed is, "Please separate the caps as burnable waste."

[0565] Specific examples

[0566] Photo and token example:

[0567] The user takes a photo of the plastic bottle after drinking and sends it through the app. The server identifies it as a plastic bottle. The user is given 5 points along with the instruction to "separate the plastic bottle as plastic waste."

[0568] Example prompt sentence:

[0569] "You take a photo of the waste and submit it. The server identifies it as a plastic bottle. It shows you how to sort it properly and awards you 5 points."

[0570] This system allows users to earn points by properly separating waste and use these points for electronic payment services, making it easier to practice environmental conservation activities in everyday life.

[0571] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0572] Step 1:

[0573] The user takes an image of the waste. The input is the image data taken by the user, and the output is that the image is saved on the device. The user takes a photo of the waste using the camera function of their smartphone or tablet. This data is saved in the device's memory as a raw image file.

[0574] Step 2:

[0575] The device sends the captured image data to the server. The input is the image data stored on the device, and the output is the image data sent to the server via the Internet. The device application uses the Internet connection to send the image data to the server's API endpoint as a POST request.

[0576] Step 3:

[0577] The server analyzes the received image data. The input is the image data sent to the server, and the output is the type of waste determined by image analysis. The server uses image analysis algorithms such as TensorFlow to process the image data and determine the type of waste.

[0578] Step 4:

[0579] The server generates a sorting method based on the classification results. The input is the type of waste identified, and the output is a text message describing the appropriate sorting method. For example, for PET bottles, the server generates the message "Please separate PET bottles as plastic waste."

[0580] Step 5:

[0581] The server sends the generated classification method to the user's terminal and displays it. The input is a text message of the generated classification method, and the output is the text message displayed on the user's terminal. The server sends this message to the terminal, and the terminal application displays the message on the screen.

[0582] Step 6:

[0583] The server awards points according to waste sorting. The input is confirmation of the sorting method, and the output is the awarded points. After confirming that the user has performed the specified sorting, the server adds the points to the user's account.

[0584] Step 7:

[0585] The server updates the point information to the user's terminal in real time. The input is the awarded point information, and the output is a text message of the updated point balance. For example, a message such as "5 points have been awarded. Your current point balance is 20 points" is displayed.

[0586] Step 8:

[0587] A user browses environmental education content. The input is the user's browsing behavior, and the output is the viewed content information. The user browses videos and articles about recycling on their device. This data is sent to the server.

[0588] Step 9:

[0589] The server checks the viewing information and awards points. The input is the viewing completion information, and the output is the awarded points. The server checks that the user has viewed the content, and adds points to the user based on that information.

[0590] Step 10:

[0591] The user enters a question about the sorting method. The input is the question entered by the user, and the output is the question data sent. The user enters the question using the chat function in the app and sends the data to the server.

[0592] Step 11:

[0593] The server answers questions using a natural language processing algorithm. The input is the question data sent by the user, and the output is the generated answer text. The server analyzes the question using natural language processing, generates an appropriate answer, and sends it to the terminal. An answer such as "Please separate the caps as burnable waste" is displayed.

[0594] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0595] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0596] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0597] [Second embodiment]

[0598] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0599] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0600] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0601] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0602] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0603] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0604] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0605] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0606] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0607] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0608] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0609] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0610] The present invention is a system that supports users in taking photos of waste and sorting it appropriately. This system is composed of components such as a terminal, a server, and a user.

[0611] System Overview

[0612] 1. Taking photos of trash and sending the data

[0613] Users take photos of waste using their devices (smartphones or tablets), and the devices send the photo data to the server. For example, if a user takes a photo of a plastic bottle, the photo data is sent to the server.

[0614] 2. Image analysis on the server

[0615] The server passes the received photo data to an image analysis algorithm, which determines the type of waste (e.g., plastic bottle, paper, metal, etc.) based on its shape and characteristics. After analysis, the server confirms that the waste is a plastic bottle.

[0616] 3. Presentation of sorting methods

[0617] Based on the analysis results, the server generates a sorting method appropriate for the PET bottles. For example, it generates a message such as "Please separate PET bottles as plastic waste." This message is sent to the terminal and displayed to the user.

[0618] 4. Real-time chat support

[0619] If the user has any further questions about the sorting method, they can use the chat function on their device to input their question. For example, they can input, "How should I dispose of this bottle cap?" The device sends this question to the server, which then analyzes the question using a natural language processing algorithm and generates an answer. The answer generated, "Please sort the cap as burnable waste," is sent to the device and displayed to the user.

[0620] 5. Environmental education and point redemption

[0621] The user views environmental education content (e.g., videos or articles about recycling). The device sends the viewing information to the server. The server confirms that the viewing has been completed and awards points to the user based on that information. The point information is updated in real time on the user's device and displayed. For example, it may say, "10 points awarded."

[0622] 6. Community Participation and Support

[0623] Users join online communities and participate in discussions and events related to environmental protection. The device notifies the server of their participation. The server records the participation information and awards rewards and badges as necessary. This information is also sent to the device and displayed to the user. For example, the server may notify the user that they have earned a recycling master badge.

[0624] Specific examples

[0625] Example 1: Taking photos of garbage and showing how to separate it

[0626] The user takes a photo of a plastic bottle with their device and sends the data to the server. The server recognizes it as a plastic bottle and generates a message saying "Please separate plastic bottles as plastic waste" and sends it to the device. The message is displayed on the user's device, allowing the user to separate the bottle correctly.

[0627] Example 2: Watching environmental education videos and earning points

[0628] The user watches a video about recycling on their device, and after completing the viewing, the information is sent to the server. The server confirms the completion of the viewing and awards the user 10 points. The user's device displays "10 points awarded," encouraging environmental education.

[0629] This is a support system that helps users easily separate their garbage and raise environmental awareness. It is a powerful tool for achieving sustainable living by providing real-time advice, reward points, and encouraging community participation.

[0630] The processing flow will be explained below.

[0631] Step 1:

[0632] The user launches the camera app on the device and takes a photo of the waste (e.g., a plastic bottle).

[0633] Step 2:

[0634] The device temporarily stores the captured photo data and prepares an HTTP POST request to send it to the server.

[0635] Step 3:

[0636] The device sends the photo data to the server via an HTTP POST request.

[0637] Step 4:

[0638] The server receives the HTTP POST request and retrieves the photo data.

[0639] Step 5:

[0640] The server uses an AI model for image analysis to analyze the photo data and determine the type of waste, using feature extraction and identification algorithms.

[0641] Step 6:

[0642] Based on the analysis results, the server generates an appropriate method for separating waste according to the type of waste. For example, it generates a message such as "Please separate PET bottles as plastic waste."

[0643] Step 7:

[0644] The server constructs a response in JSON format that includes the generated sorting method and sends it to the terminal.

[0645] Step 8:

[0646] The terminal receives and analyzes the response from the server.

[0647] Step 9:

[0648] The terminal displays the analyzed sorting method to the user.

[0649] Step 10:

[0650] If the user has any further questions about the sorting method, they can use the chat function on the device to type in their questions.

[0651] Step 11:

[0652] The terminal sends the user's question to the server.

[0653] Step 12:

[0654] The server receives the question and uses natural language processing (NLP) algorithms to analyze the question and generate an answer.

[0655] Step 13:

[0656] The server sends the generated response to the terminal.

[0657] Step 14:

[0658] The terminal receives the response and displays it to the user.

[0659] Step 15:

[0660] A user selects environmental education content (e.g., a video about recycling) on ​​the device to watch.

[0661] Step 16:

[0662] The terminal notifies the server of the user's viewing actions.

[0663] Step 17:

[0664] When the user finishes watching the video, the device sends that information to the server.

[0665] Step 18:

[0666] The server confirms that the viewing has been completed and awards points to the user.

[0667] Step 19:

[0668] The server transmits the awarded point information to the terminal.

[0669] Step 20:

[0670] The terminal displays the updated points information to the user.

[0671] Step 21:

[0672] When a user participates in a discussion or event in an online community, the terminal notifies the server of the participation information.

[0673] Step 22:

[0674] The server records participation information and awards rewards and badges to users as appropriate.

[0675] Step 23:

[0676] The server sends reward and badge information to the terminal.

[0677] Step 24:

[0678] The terminal displays reward and badge information to the user.

[0679] Example 1

[0680] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0681] In conventional waste sorting systems, users have difficulty accurately identifying the type of waste and rarely have detailed knowledge of how to sort it. Furthermore, existing systems make it difficult for users to receive ongoing environmental education or community activity support, resulting in insufficient improvement in users' environmental awareness. Therefore, there is a need for a system that promotes sustainable lifestyles by supporting proper waste sorting and providing users with ongoing environmental education and community activities.

[0682] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0683] In this invention, the server includes: a means for a user to acquire an image of waste; a means for transmitting image data from a terminal to the server; a means for the server to analyze the received image data and identify the type of waste; a means for generating a recommended sorting method based on the type of waste identified by the server; a means for transmitting the generated sorting method to the terminal and displaying it; a means for the user to input additional questions and transmit them from the terminal to the server; a means for the server to analyze the questions and generate answers using natural language processing; and a means for transmitting the generated answers to the terminal and displaying them. This allows users to easily and accurately sort waste and obtain necessary information in real time. Furthermore, the system includes a means for transmitting information about viewing environmental education content to the server and awarding points, and a means for recording participation information in the online community and awarding rewards and badges, thereby improving users' environmental awareness and supporting sustainable lifestyles.

[0684] "User" refers to an individual who uses the system to perform operations related to waste separation and environmental education.

[0685] A "terminal" is a communication device used by a user, and specifically includes a smartphone, tablet, computer, etc.

[0686] A "server" is a computing device that receives and processes data sent from a terminal, and is responsible for analyzing the data and generating results.

[0687] "Image data" refers to photographic information of waste taken by a user using a terminal.

[0688] "Image analysis" refers to the computational process in which the server processes image data to determine the type of waste.

[0689] "Sorting methods" refer to specific procedures and rules for properly classifying and disposing of waste.

[0690] "Natural language processing" refers to a technology that allows a server to understand questions or messages entered by users and generate appropriate answers.

[0691] "Environmental education content" refers to teaching materials and resources that users can view to increase their environmental knowledge and awareness.

[0692] "Points" refer to rewards given to users for viewing environmental education content or using the system.

[0693] "Online community" refers to a network group in which users can participate to exchange information and engage in activities related to environmental protection.

[0694] "Rewards" refers to incentives such as badges and points that users receive through their activities in online communities.

[0695] "Badge" refers to a digital award or recognition given to users to recognize their achievements in online communities.

[0696] The present invention is a system that supports users in separating waste materials easily and appropriately. This system is composed of elements such as terminals, servers, and users. Specifically, it operates as follows.

[0697] Taking photos of trash and sending the data

[0698] Users take photos of waste using a device (e.g., a smartphone or tablet). The captured photo data is sent from the device to a server. This transmission is performed using a REST API using the HTTP protocol.

[0699] Image analysis on the server

[0700] The server passes the received image data to an image analysis algorithm. Specific algorithms used include image classification models such as TensorFlow and OpenCV. The image analysis algorithm identifies the shape and characteristics of the waste and identifies its type (e.g., plastic bottles, paper, metal, etc.).

[0701] Presentation of sorting methods

[0702] The server generates an appropriate sorting method based on the results of image analysis. For example, if it identifies a plastic bottle, it generates a message saying, "Please separate plastic bottles as plastic waste." This message is sent from the server to the terminal and displayed to the user.

[0703] Real-time chat support

[0704] If the user has any additional questions about the sorting method, they can enter them using the chat function on their device. This question is then sent from the device to the server. The server then analyzes the received question using a natural language processing algorithm (e.g., GPT-3) and generates an appropriate answer. The generated answer is then sent to the device and displayed to the user.

[0705] Environmental education and point redemption

[0706] The user views environmental education content (for example, videos or articles about recycling) on ​​the device. The viewing information is sent from the device to the server, which then confirms that the viewing is complete and awards points to the user. For example, a message saying "10 points awarded" is displayed on the device.

[0707] Community participation and support

[0708] Users join online communities and participate in discussions and events related to environmental protection. Participation information is sent from the device to the server, which records the participation information and awards rewards and badges as necessary. For example, a message saying "You have earned the Recycling Master badge" is displayed on the device.

[0709] Specific examples

[0710] Example 1: Taking photos of garbage and showing how to separate it

[0711] A user takes a photo of a plastic bottle with their device and sends the data to the server. The server recognizes it as a plastic bottle, generates a message saying "Please separate plastic bottles as plastic waste," and sends it to the device. The message is displayed on the user's device, allowing the user to separate the bottles correctly.

[0712] Example of a generative AI model input prompt: "I took a photo of some waste. How should I sort it?"

[0713] Example 2: Watching environmental education videos and earning points

[0714] A user watches a video about recycling on their device, and after completing the viewing, the information is sent to the server. The server confirms the completion of the viewing and awards the user 10 points. The user's device displays "10 points awarded," encouraging environmental education.

[0715] Example of a generative AI model input prompt: "I watched the recycling video to the end. Please give me points."

[0716] By combining these functions, the system helps users separate waste accurately and efficiently, and increases environmental awareness. It supports the development of sustainable lifestyles through real-time advice, reward points, and community participation.

[0717] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0718] Step 1: User takes a photo on their device

[0719] Users take photos of waste using devices such as smartphones or tablets. The input for this process is the image of waste taken by the user using the device's camera function, and the output is image data. Specifically, the user launches the camera app, points it at the waste, and presses the shutter button.

[0720] Step 2: The device sends the photo data to the server

[0721] The device sends the captured photo data to the server. The input to this process is the image data stored on the device, and the output is the transfer of the image data to the server. A REST API using the HTTP protocol is used to send the data. Specifically, the application on the device generates an HTTP request containing the image data and sends it to the specified server endpoint.

[0722] Step 3: The server receives the image data.

[0723] The server receives image data sent from the terminal. The input of this process is image data sent via the network, and the output is image data saved on the server. Specifically, the server receives an HTTP request and temporarily saves the image data.

[0724] Step 4: The server performs image analysis

[0725] The server passes the image data to an image analysis algorithm. The input for this process is the image data stored on the server, and the output is an analysis result indicating the type of waste. Specifically, the image data is analyzed using an image classification model using TensorFlow and OpenCV to obtain classification results such as PET bottles, paper, and metal.

[0726] Step 5: The server generates the sorting method

[0727] The server generates a sorting method based on the results of image analysis. The input to this process is the result of image analysis, and the output is a message indicating the sorting method. Specifically, it generates a message such as "Please separate PET bottles as plastic waste" based on a predefined database and rules.

[0728] Step 6: The server sends a classification method message to the terminal.

[0729] The server sends the generated sorting method message to the terminal. The input of this process is the generated sorting method message, and the output is message delivery to the terminal. Specifically, the server sends the message to the terminal using an HTTP response or a messaging queue.

[0730] Step 7: The device displays a sorting method message to the user.

[0731] The terminal displays the received message on the sorting method to the user. The input of this process is the message on the sorting method sent from the server, and the output is the message displayed on the terminal screen. Specifically, the terminal application receives the message and displays it on the user interface (UI).

[0732] Step 8: User enters follow-up question into terminal

[0733] The user enters additional questions about the sorting method using the chat function on the device. The input for this process is the user's question text, and the output is the question data entered into the device. Specifically, the user opens a chat app and enters, "How should I dispose of this bottle cap?"

[0734] Step 9: The device sends the query data to the server

[0735] The terminal sends the entered question data to the server. The input of this process is the question data entered into the terminal, and the output is the transfer of the question data to the server. Specifically, the terminal generates an HTTP request including the question data and sends it to the server.

[0736] Step 10: The server performs natural language processing and generates an answer

[0737] The server analyzes the received question data using a natural language processing algorithm and generates an answer. The input for this process is the question data, and the output is the generated answer text. Specifically, it uses a natural language processing model such as GPT-3 to generate the answer "Please separate the caps as burnable garbage."

[0738] Step 11: The server sends the response data to the terminal.

[0739] The server sends the generated answer data to the terminal. The input of this process is the generated answer text, and the output is the transfer of the answer data to the terminal. Specifically, the server generates an HTTP response including the answer data and sends it to the terminal.

[0740] Step 12: The device displays the response data to the user

[0741] The terminal displays the received answer data to the user. The input of this process is the answer text sent from the server, and the output is the answer displayed on the terminal screen. Specifically, the terminal application receives the answer data and displays it on the user interface (UI).

[0742] In this way, the system helps users separate waste accurately and easily, provides additional information in real time, and provides a mechanism to increase users' environmental awareness through environmental education and community activities.

[0743] (Application example 1)

[0744] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0745] In recent years, the increase in the use of food delivery services has led to a significant increase in waste, including containers and packaging materials. While this has created a demand for proper waste separation, users find it difficult to find the time to do so due to their busy daily lives. Furthermore, the complicated separation method often leads to incorrect separation. Furthermore, the lack of incentives for separating waste often leads to a decline in motivation. A system is needed to solve these problems, encourage users to separate waste effectively, and raise their environmental awareness.

[0746] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0747] In this invention, the server includes means for a user to take a photo of waste, means for transmitting the photo data to the server, means for the server to analyze the photo data and determine the type of waste, means for generating a recommended sorting method based on the type of waste determined by the server, means for transmitting the sorting method generated by the server to the user's terminal and displaying it, means for the user to be awarded points according to their waste sorting and displaying the point information, and means for the user to view environmental education content and transmit the viewed information to the server. This allows users to easily learn the appropriate sorting method for food delivery waste, increasing their incentive to sort waste and improving overall environmental awareness.

[0748] "User" refers to a person who uses the system to separate waste.

[0749] "Waste" refers to unwanted items and garbage.

[0750] "Photo" refers to image data that a user takes to record the state of waste.

[0751] "Means" refers to the methods or devices that a system uses to achieve its goals.

[0752] "Server" refers to a central management device that receives and analyzes data sent by users, and generates and transmits the necessary information.

[0753] "Data" refers collectively to information processed within the system, such as photos taken by users and information generated by the system.

[0754] "Analysis" refers to the process by which the server processes the received photo data and identifies the type of waste.

[0755] "Sorting methods" refers to instructions and guidelines for properly disposing of and classifying waste.

[0756] "Terminal" refers to the device (such as a smartphone or tablet) that a user uses to access the system.

[0757] "Points" refer to rewards given to users for properly sorting waste and viewing environmental education content.

[0758] "Point information" refers to information indicating the status and history of points given to a user.

[0759] "Environmental education content" refers to educational materials such as videos and articles that help users increase their environmental awareness through learning.

[0760] This invention is a system that supports the separation of waste generated by food delivery. The system allows users to take photos of waste and send them to a server, which then presents the user with appropriate separation methods and provides a function that awards points for following the separation methods. The following describes in detail an embodiment of this invention.

[0761] 1. Hardware and Software Requirements

[0762] Hardware: Smartphones, servers

[0763] Software: OpenCV, Scikit-learn, Joblib, Natural Language Processing algorithms

[0764] 2. System Configuration

[0765] User's device: The user takes a photo of the waste using a smartphone or tablet, and the device sends the photo data to the server via the Internet.

[0766] Server: The server passes the received photo data to an image analysis algorithm to identify the type of waste. Based on the analysis results, it generates a recommended sorting method and sends it to the user's device. It also awards points based on the user's behavior and manages the points history.

[0767] Image analysis algorithm: Images are preprocessed using OpenCV, and waste type is identified using a Scikit-learn-based model that has been trained on image data of various waste materials.

[0768] Natural language processing algorithms: When users enter a question, the system analyzes the question and generates an appropriate answer, allowing users to receive real-time support on how to sort in detail.

[0769] 3. System Operation

[0770] Users take photos of waste generated during food delivery and send them to a server via the app. The server then passes the received photos through an image analysis algorithm to determine the type of waste. For example, if the waste is a plastic container, a message saying "Please separate it as plastic waste" is generated and displayed on the user's device. When users follow the sorting instructions, they are awarded points based on their actions, which can be used as incentives such as discounts on delivery.

[0771] Specific examples

[0772] 1. Taking photos of trash and showing how to separate it

[0773] The user takes a photo of the food container after delivery with their smartphone and sends it to the server via the app. The server analyzes the photo and displays a message to the user saying, "Please separate this plastic container as burnable garbage."

[0774] 2. Points Award

[0775] When users correctly separate their trash according to the provided separation method, the app's internal history is updated and they are awarded recycling points, which can be used for discounts on their next food delivery.

[0776] Prompt Sentence Examples

[0777] User Question: "Which garbage bag should I put this plastic container in?"

[0778] Prompt for generative AI model: "Please tell me the proper way to separate plastic containers."

[0779] User Question: "How do I dispose of paper bags?"

[0780] Prompt for generative AI model: "How do I dispose of paper bags?"

[0781] In this way, the present invention becomes a powerful tool to help users properly separate waste and promote environmental awareness.

[0782] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0783] Step 1:

[0784] The user takes a photo of the waste used in food delivery using their smartphone. To do this, the user launches the application and takes a photo using the camera function. The input is the photo of the waste taken. The output is that this photo data is saved on the device.

[0785] Step 2:

[0786] The device initiates network communication to send the captured photo data to the server. The input is the photo data, and the output is the photo data sent to the server. A secure communication method such as HTTPS is used to send the data.

[0787] Step 3:

[0788] The server passes the received photo data to the image analysis algorithm, which uses OpenCV to pre-process the photos. This pre-processing involves resizing and normalizing the photos to convert the image data into an analyzable format. The input is the photo data sent to the server, and the output is the pre-processed image data.

[0789] Step 4:

[0790] The server passes the preprocessed image data to a waste classification model. Here, a pre-trained model from Scikit-learn is used. The model analyzes the image data and identifies the type of waste. The input is the preprocessed image data, and the output is the analysis result. For example, it may be identified as a plastic container.

[0791] Step 5:

[0792] The server generates a recommended sorting method based on the analysis results. The input is the type of waste (analysis result), and the output is a message about the sorting method. For example, a message such as "Please sort this plastic container as plastic waste" is generated.

[0793] Step 6:

[0794] The server sends the generated message of the sorting method to the terminal. The input is the generated message of the sorting method, and the output is the message sent to the terminal. The message instructs the user on the sorting method.

[0795] Step 7:

[0796] The terminal displays the message of the sorting method received from the server to the user. The user checks the displayed message and understands the sorting method. The input is the message received from the server, and the output is the message displayed to the user.

[0797] Step 8:

[0798] The user must confirm the operation before points can be awarded for sorting waste. After the user completes the sorting task, the user sends the information to the server via their device. The input is the user's sorting task completion information, and the output is the information sent to the server.

[0799] Step 9:

[0800] The server receives the user's sorting completion information and awards points. The input is the user's sorting completion information, and the output is the points awarded to the user. The points can be used as a discount for the next delivery.

[0801] Step 10:

[0802] The server receives information about the user's browsing of environmental education content and awards points based on that information. The input is the user's browsing information of environmental education content, and the output is the point information awarded to the user.

[0803] Step 11:

[0804] When a user uses a discount or benefit, the point information is checked on the terminal. The terminal displays the user's point balance and history. The input is the user's point information, and the output is the point information displayed on the terminal.

[0805] Through the above steps, a system is realized in which users can properly separate their waste and earn points to receive a discount on their next food delivery.

[0806] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0807] This invention is a system that supports users in taking photos of waste and sorting it appropriately. This system consists of components: a terminal, a server, a user, and an emotion engine.

[0808] System Overview

[0809] 1. Taking photos of trash and sending the data

[0810] Users take photos of waste using their devices (smartphones or tablets), and the devices send the photo data to the server. For example, if a user takes a photo of a plastic bottle, the photo data is sent to the server.

[0811] 2. Image analysis on the server

[0812] The server passes the received photo data to an image analysis algorithm, which determines the type of waste (e.g., plastic bottle, paper, metal, etc.) based on its shape and characteristics. After analysis, the server confirms that the waste is a plastic bottle.

[0813] 3. Presentation of sorting methods

[0814] Based on the analysis results, the server generates a sorting method appropriate for the PET bottles. For example, it generates a message such as "Please separate PET bottles as plastic waste." This message is sent to the terminal and displayed to the user.

[0815] 4. Real-time chat support

[0816] If the user has any further questions about the sorting method, they can use the chat function on their device to input their question. For example, they can input, "How should I dispose of this bottle cap?" The device sends this question to the server, which then analyzes the question using a natural language processing algorithm and generates an answer. The answer generated, "Please sort the cap as burnable waste," is sent to the device and displayed to the user.

[0817] 5. Environmental education and point redemption

[0818] The user views environmental education content (e.g., videos or articles about recycling). The device sends the viewing information to the server. The server confirms that the viewing has been completed and awards points to the user based on that information. The point information is updated in real time on the user's device and displayed. For example, it may say, "10 points awarded."

[0819] 6. Community Participation and Support

[0820] Users join online communities and participate in discussions and events related to environmental protection. The device notifies the server of their participation. The server records the participation information and awards rewards and badges as necessary. This information is also sent to the device and displayed to the user. For example, the server may notify the user that they have earned a recycling master badge.

[0821] 7. Improving user experience by incorporating an emotion engine

[0822] The emotion engine has the ability to analyze a user's facial expressions and voice and recognize their emotions. For example, when a user takes a photo or types a question using the chat function, the facial expression and voice data are acquired. The device sends this data to the server, and the server uses the emotion engine to recognize the user's emotions.

[0823] The recognized emotion information is taken into consideration when presenting methods and answers. For example, if the user is confused, a more detailed and easy-to-understand explanation is presented. If the user is satisfied, a simple message is displayed.

[0824] Specific examples

[0825] Example 1: Taking photos of garbage and showing how to separate it

[0826] The user takes a photo of a plastic bottle with their device and sends the data to the server. The server recognizes it as a plastic bottle and generates a message saying "Please separate plastic bottles as plastic waste" and sends it to the device. The message is displayed on the user's device, allowing the user to separate the bottles correctly. The user's facial expression at this time is also recorded and sent as data to the server.

[0827] Example 2: Watching environmental education videos and earning points

[0828] The user watches a video about recycling on their device, and after watching, the information is sent to the server. The server confirms that the user has watched the video and awards the user 10 points. The user's device displays a message saying "10 points awarded," encouraging them to become more environmentally educated. The user's emotions are also recorded, and if the user's satisfaction after watching is high, this information is reflected in the next content suggestions.

[0829] This invention is a support system that helps users easily separate their garbage and raise environmental awareness. It is a powerful tool for achieving sustainable living by providing real-time advice, reward points, and encouraging community participation. Furthermore, the incorporation of an emotion engine improves the user experience and provides more effective education and support.

[0830] The processing flow will be explained below.

[0831] Step 1:

[0832] The user launches the camera app on the device and takes a photo of the waste (e.g., a plastic bottle).

[0833] Step 2:

[0834] The device temporarily stores the captured photo data, acquires the user's facial expression data, and prepares an HTTP POST request to send to the server.

[0835] Step 3:

[0836] The device sends the photo data and facial expression data to the server via an HTTP POST request.

[0837] Step 4:

[0838] The server receives the HTTP POST request and retrieves the photo data and facial expression data.

[0839] Step 5:

[0840] The server uses an AI model for image analysis to analyze the photo data and determine the type of waste, using feature extraction and identification algorithms.

[0841] Step 6:

[0842] The server uses an emotion engine to analyze the facial expression data and recognize the user's emotional state (e.g., joy, confusion, indifference, etc.).

[0843] Step 7:

[0844] Based on the analysis results, the server generates an appropriate method for separating waste according to the type of waste. The content and tone of the message are adjusted depending on the user's emotional state. For example, for a confused user, the server generates a detailed explanation such as "Please separate PET bottles as plastic waste. Please see below for detailed instructions."

[0845] Step 8:

[0846] The server constructs a response in JSON format that includes the generated sorting method and sends it to the terminal.

[0847] Step 9:

[0848] The terminal receives and analyzes the response from the server.

[0849] Step 10:

[0850] The device displays the analyzed classification method to the user, and also displays additional support messages depending on the user's emotional state.

[0851] Step 11:

[0852] If the user has any further questions about the sorting method, they can use the chat function on the device to type in their questions.

[0853] Step 12:

[0854] The terminal transmits the user's question together with the reacquired facial expression data to the server.

[0855] Step 13:

[0856] The server receives the question and facial expression data, analyzes the question using natural language processing (NLP) algorithms, and reconfirms the user's emotional state.

[0857] Step 14:

[0858] The server generates an appropriate response, again adjusting it based on the emotional state, for example adding a follow-up message such as "Please separate the caps as burnable trash. If you have any further questions, please let us know."

[0859] Step 15:

[0860] The server sends the generated response to the terminal.

[0861] Step 16:

[0862] The terminal receives the response and displays it to the user.

[0863] Step 17:

[0864] A user selects environmental education content (e.g., a video about recycling) on ​​the device to watch.

[0865] Step 18:

[0866] The terminal notifies the server of the user's selection and facial expression data.

[0867] Step 19:

[0868] When the user finishes watching the video, the terminal transmits viewing completion information and facial expression data during viewing to the server.

[0869] Step 20:

[0870] The server confirms that the viewing has been completed and awards points to the user.

[0871] Step 21:

[0872] The server takes into consideration the points awarded and the user's emotional state while watching, adjusts the next recommended content, and sends it to the terminal.

[0873] Step 22:

[0874] The device displays updated point information and recommended content tailored to the user's emotional state.

[0875] Step 23:

[0876] When a user participates in a discussion or event in an online community, the terminal notifies the server of the participation information.

[0877] Step 24:

[0878] The server records participation information and awards rewards and badges to users as appropriate.

[0879] Step 25:

[0880] The server sends reward and badge information to the terminal.

[0881] Step 26:

[0882] The device displays reward and badge information to the user, and also collects emotional data from the user during the event to reflect in the next event information.

[0883] Example 2

[0884] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0885] Currently, waste sorting is generally done manually, making it difficult for users to consistently and accurately understand and follow the sorting method. Furthermore, when users have questions about the sorting method, they often cannot immediately resolve them, preventing them from sorting accurately. Furthermore, there is a lack of support to maintain motivation for waste sorting and flexible support based on users' emotions. There are also limited incentives for viewing environmental education content, making it necessary to improve eco-consciousness.

[0886] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0887] In this invention, the server includes: a device that transmits photo data and additional data to the server; a server that passes the photo data received from the device to an image analysis algorithm to identify the type of waste; a device that generates a recommended sorting method based on the type of waste identified by the server; a device that transmits the generated sorting method to the terminal and displays it to the user; a device that acquires the user's facial expression and voice data and transmits it to the server; and a server that recognizes the user's emotions using an emotion analysis algorithm and uses that information to present the sorting method. This allows users to easily and accurately sort waste and improves the user experience by providing real-time question and answer services and flexible support based on emotions. Furthermore, a points system for viewing environmental education content can improve eco-consciousness and promote sustainable behavior.

[0888] "User" refers to an individual or organization that uses this system to separate waste.

[0889] "Terminal" refers to an electronic device such as a smartphone or tablet used by a user.

[0890] A "server" refers to a computer system that receives and analyzes data sent from a terminal and sends the results back to the terminal.

[0891] "Photo data" refers to image data of waste photographed by a user using a terminal.

[0892] "Additional data" refers to additional information such as GPS data and timestamps that are acquired along with the photo data.

[0893] "Image analysis algorithm" refers to a machine learning algorithm that analyzes received photo data and determines the type of waste.

[0894] "Waste type" refers to the category into which waste is classified, examples include plastic bottles, paper, metal, etc.

[0895] "Sorting method" refers to the appropriate disposal method recommended to the user based on the identified type of waste.

[0896] "Facial expression data" refers to data relating to the user's facial expressions.

[0897] "Voice data" refers to voice data relating to the content of a user's speech.

[0898] An "emotion analysis algorithm" refers to an algorithm that analyzes facial expression data and voice data to estimate a user's emotions.

[0899] A "natural language processing algorithm" refers to an algorithm that analyzes a user's question or input text and generates an appropriate answer.

[0900] "Environmental education content" refers to information such as videos and articles that are intended to educate people about recycling and environmental protection.

[0901] "Points" refer to rewards given to users for viewing environmental education content and engaging in proper waste sorting activities.

[0902] The system of the present invention assists users in taking photos of waste and sorting it appropriately. This system is implemented by users using devices such as smartphones or tablets, which communicate with a server. The system also incorporates an emotion engine and natural language processing algorithms to improve the user experience.

[0903] Hardware and Software

[0904] 1. Terminal

[0905] Electronic devices such as smartphones and tablets.

[0906] Includes a camera, GPS module, microphone, etc.

[0907] 2. Server

[0908] A computer system that processes and analyzes data.

[0909] Implemented database, image analysis algorithm, natural language processing algorithm, and sentiment analysis algorithm.

[0910] 3. Software

[0911] Image analysis algorithms (e.g. TensorFlow)

[0912] Natural language processing algorithms (e.g., OpenAI's GPT-3)

[0913] Sentiment analysis algorithms (e.g., Microsoft's Azure Cognitive Services)

[0914] Data processing and calculation

[0915] 1. Taking photos of trash and sending the data

[0916] The user uses the terminal to take a photo of the waste.

[0917] The device sends the photo data along with GPS data and a timestamp to the server.

[0918] 2. Image analysis on the server

[0919] The server passes the received photo data to an image analysis algorithm.

[0920] The algorithm identifies the type of waste (plastic bottles, paper, metal, etc.).

[0921] 3. Presentation of sorting methods

[0922] The server generates an appropriate sorting method based on the results of image analysis.

[0923] The server sends the generated sorting method to the terminal and displays it to the user.

[0924] 4. Real-time chat support

[0925] The user enters a question using the chat function of the device.

[0926] The server uses natural language processing algorithms to analyze the question and generate an answer.

[0927] The generated answer is sent to the terminal and displayed to the user.

[0928] 5. Environmental education and point redemption

[0929] The user views the environmental education content on the device.

[0930] The terminal transmits the browsing information to the server.

[0931] The server confirms that the viewing is complete and gives points to the user.

[0932] Points information is displayed on the device in real time.

[0933] 6. Community Participation and Support

[0934] Users participate in online communities and engage in activities.

[0935] The terminal transmits the participation information to the server.

[0936] The server will award rewards and badges based on activity records.

[0937] Reward information will be displayed on the device.

[0938] 7. Use of Emotion Engines

[0939] When the device takes a photo or uses the chat function, it collects the user's facial expressions and voice data.

[0940] The terminal transmits this data to the server.

[0941] The server analyzes the user's emotions using an emotion analysis algorithm.

[0942] Based on the analysis results, sorting methods and responses are individually adjusted to improve the user experience.

[0943] Specific examples

[0944] Example 1: Taking photos of garbage and showing how to separate it

[0945] The user takes a photo of the plastic bottle with the device and sends the data to the server.

[0946] The server recognizes it as a plastic bottle, generates a message saying "Please separate plastic bottles as plastic waste," and sends it to the terminal.

[0947] The message will be displayed on the user's terminal, allowing the user to properly distinguish between the two.

[0948] The user's facial expression at this time is also recorded and sent as data to the server.

[0949] Example prompt sentence:

[0950] Photo of a plastic bottle. How should I dispose of it?

[0951] Example 2: Watching environmental education videos and earning points

[0952] Users watch videos about recycling on their devices, and after they finish watching, the information is sent to the server.

[0953] The server confirms that the viewing has been completed and awards the user 10 points.

[0954] The user's device will display the message "10 points awarded," encouraging environmental education.

[0955] The user's emotions are also recorded at this time, and if the user is highly satisfied after viewing, this information is reflected in the next content suggestions.

[0956] Example prompt sentence:

[0957] I have completed watching the recycling video. How many points did I earn?

[0958] This invention is a support system that helps users easily separate their garbage and raise environmental awareness. It is a powerful tool for achieving sustainable living by providing real-time advice, reward points, and encouraging community participation. Furthermore, the incorporation of an emotion engine improves the user experience and provides more effective education and support.

[0959] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0960] Step 1:

[0961] Garbage photography

[0962] Operation:

[0963] Users take photos of waste using the camera function on their device, such as a smartphone or tablet.

[0964] input:

[0965] The physical object of waste (e.g., plastic bottles).

[0966] output:

[0967] Save as image data on the device.

[0968] Specific operation:

[0969] The user opens the camera app, points the lens at the waste, and presses the shutter button. The device takes a photo and saves the image data to the internal storage. GPS data and a timestamp are also acquired.

[0970] Step 2:

[0971] Sending photo data

[0972] Operation:

[0973] The device sends the photo data and additional data (GPS data, timestamp) to the server.

[0974] input:

[0975] Image data, GPS data, timestamp.

[0976] output:

[0977] Data transmission to the server completed.

[0978] Specific operation:

[0979] The user presses the "Send" button in the application on the device. The device uploads the photo data and additional data to the server via the Internet. The server receives the data and temporarily stores it in a database.

[0980] Step 3:

[0981] Receiving and storing image data

[0982] Operation:

[0983] The server receives the photo data and additional data sent from the terminal and temporarily stores them.

[0984] input:

[0985] Image data, GPS data, timestamp.

[0986] output:

[0987] Storing data in a database.

[0988] Specific operation:

[0989] When the server receives the data sent from the terminal, it temporarily stores the image data and additional data in the database, and creates a new entry in the database according to the existing data structure.

[0990] Step 4:

[0991] Starting image analysis

[0992] Operation:

[0993] The server passes the received photo data to an image analysis algorithm to determine the type of waste.

[0994] input:

[0995] Image data.

[0996] output:

[0997] Analysis results (type of waste).

[0998] Specific operation:

[0999] The server invokes an image analysis algorithm (e.g., TensorFlow) to analyze the stored image data. The algorithm extracts features and identifies the type of waste in the image. The results are returned as text, such as "PET bottle."

[1000] Step 5:

[1001] Generation of separation methods

[1002] Operation:

[1003] The server generates an appropriate classification method based on the analysis results.

[1004] input:

[1005] Analysis results (type of waste).

[1006] output:

[1007] Message on how to sort waste.

[1008] Specific operation:

[1009] The server selects a message template based on the type of waste and generates specific sorting instructions, such as "Please separate PET bottles as plastic waste." The generated message is temporarily stored.

[1010] Step 6:

[1011] Sending sorting instructions

[1012] Operation:

[1013] The server transmits the generated sorting method to the terminal.

[1014] input:

[1015] Message on how to sort waste.

[1016] output:

[1017] Message sent to device completed.

[1018] Specific operation:

[1019] The server sends a message including the classification method to the terminal. This message reaches the terminal via the Internet. The terminal notifies the user of the received message.

[1020] Step 7:

[1021] Display of sorting method

[1022] Operation:

[1023] The terminal displays the received sorting method to the user.

[1024] input:

[1025] Message on how to sort waste.

[1026] output:

[1027] Displayed on the user screen.

[1028] Specific operation:

[1029] The device will then display the received sorting instructions in a pop-up notification or in a message box within the app, showing the user the message, "Please separate PET bottles as plastic waste."

[1030] Step 8:

[1031] Entering user questions

[1032] Operation:

[1033] Users can use the chat function on their device to enter questions about sorting methods.

[1034] input:

[1035] User text input (question).

[1036] output:

[1037] Prepare to send question data from the terminal to the server.

[1038] Specific operation:

[1039] A user opens a chat application and types a question such as, "How should I dispose of this bottle cap?" Once the question is typed, the device prepares the data to be sent to the server.

[1040] Step 9:

[1041] Submitting Question Data

[1042] Operation:

[1043] The terminal transmits the entered question to the server.

[1044] input:

[1045] User text input (question).

[1046] output:

[1047] Question data has been sent to the server.

[1048] Specific operation:

[1049] The device sends the question data entered by the user to the server via the Internet. After sending, a message saying "Question has been sent" is displayed on the screen.

[1050] Step 10:

[1051] Question analysis and answer generation

[1052] Operation:

[1053] The server passes the question data to a natural language processing algorithm to generate an answer.

[1054] input:

[1055] User text input (question).

[1056] output:

[1057] Analysis result (answer).

[1058] Specific operation:

[1059] The server analyzes the question using a natural language processing algorithm (e.g., GPT-3). The algorithm understands the context and generates a specific answer, such as "Please separate the caps as burnable garbage." The generated answer is temporarily stored.

[1060] Step 11:

[1061] Submitting and viewing answers

[1062] Operation:

[1063] The server sends the generated answer to the terminal, which displays it to the user.

[1064] input:

[1065] Analysis result (answer).

[1066] output:

[1067] The response is sent to the device and displayed on the user's screen.

[1068] Specific operation:

[1069] The server sends a message containing the answer to the terminal, which displays the received answer in real time, showing the user the message "Please separate the caps as burnable garbage."

[1070] Step 12:

[1071] Viewing environmental education content and earning points

[1072] Operation:

[1073] The user watches the environmental education content, and the terminal sends the viewing completion information to the server. The server confirms the viewing completion and awards points.

[1074] input:

[1075] Viewing data for educational content.

[1076] output:

[1077] Points are awarded.

[1078] Specific operation:

[1079] The user watches an environmental education video, and after watching it, the device sends the information to the server. The server checks the viewing data and awards the user 10 points. The device then displays a notification to the user that points have been awarded.

[1080] Step 13:

[1081] Community Participation and Rewards

[1082] Operation:

[1083] When a user joins an online community, the terminal sends the participation information to the server, which records the participation information and grants rewards.

[1084] input:

[1085] Community participation information.

[1086] output:

[1087] Rewarding.

[1088] Specific operation:

[1089] Users join online communities and participate in discussions and events. The device sends this information to the server, which records the participation information and awards "recycling master badges" as necessary. The device then notifies the user of rewards.

[1090] Step 14:

[1091] Acquiring and analyzing emotion data

[1092] Operation:

[1093] The device captures the user's facial expressions and voice data and sends it to the server, which then analyzes their emotions using an emotion analysis algorithm.

[1094] input:

[1095] Facial expression data, voice data.

[1096] output:

[1097] Emotion analysis results.

[1098] Specific operation:

[1099] When a user takes a photo or posts a question via chat, the device uses a camera and microphone to capture facial and voice data. The device then sends this data to a server, which then runs an emotion analysis algorithm to identify the user's emotions. The analysis results are then reflected in the next step.

[1100] Step 15:

[1101] Regulating responses based on emotions

[1102] Operation:

[1103] Based on the results of the emotion analysis, the server individually adjusts the classification method and responses and sends them to the device.

[1104] input:

[1105] Emotion analysis results.

[1106] output:

[1107] Tailored response.

[1108] Specific operation:

[1109] The server generates a detailed and easy-to-understand explanation if the user is confused, or a concise message if the user is satisfied. The tailored response is sent to the terminal and displayed to the user.

[1110] (Application example 2)

[1111] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1112] Conventional waste sorting support systems help users sort waste accurately, but they lack the functionality to provide direct incentives for that behavior. Furthermore, there is no mechanism to further improve the accuracy of waste sorting and encourage users to continue using the system. As a result, users' interest and motivation declined, and participation in sustainable environmental protection activities did not expand as expected. Furthermore, because there was no point system linked to an electronic payment service, the benefits to users could not be directly linked to their daily lives.

[1113] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for a user to take an image of waste, means for transmitting image data to the server, means for the server to analyze the image data and determine the type of waste, means for generating a recommended sorting method based on the type of waste determined by the server, means for transmitting the sorting method generated by the server to the user's terminal and displaying it, and means for awarding points according to waste sorting and making the points usable for electronic payment services. This allows users to receive direct incentives by properly sorting waste, promoting continued use and expanding participation in sustainable environmental protection activities.

[1114] A "user's device" refers to an electronic device operated by a user, and mainly refers to a smartphone, tablet, or PC.

[1115] "Image data" refers to photographic data of waste that a user takes with a terminal and sends to a server.

[1116] "Server" refers to a computer system for receiving and analyzing image data via a network.

[1117] "Waste type" refers to the classification of what material or type the waste falls into (e.g., plastic bottles, paper, metal, etc.).

[1118] "Sorting method" refers to the appropriate disposal method or treatment procedure recommended based on the type of waste.

[1119] "Points" are incentives given to users who properly separate their waste, and refer to virtual currency or electronic money that can be used in electronic payment services.

[1120] "Electronic payment service" refers to an online payment system that enables users to pay for goods and services via the Internet.

[1121] "Environmental education content" refers to media such as videos, articles, and quizzes that provide users with knowledge and information about environmental protection and recycling.

[1122] "Natural language processing" refers to computer algorithms and technologies that analyze text data entered by users, understand its meaning, and generate appropriate answers.

[1123] This system allows users to take pictures of waste and properly separate it to earn points, which can then be used for electronic payment services. The system mainly consists of a user terminal, a server, an image analysis algorithm, a natural language processing algorithm, and an electronic payment platform.

[1124] 1. Use of User Devices

[1125] Users take photos of waste using devices such as smartphones or tablets. This photo data is sent to a server via the Internet. The user's device must have a camera function and be connected to the Internet.

[1126] 2. Data reception and analysis by the server

[1127] The server receives image data sent from the user's device and uses image analysis algorithms such as TensorFlow to determine the type of waste in the received image data. Depending on the type of waste, the server generates a recommended sorting method and sends it to the user's device.

[1128] 3. Display of recommended sorting methods

[1129] The user terminal receives the sorting instructions sent from the server and displays them. For example, a message such as "Please separate PET bottles as plastic waste" is displayed.

[1130] 4. Points allocation and real-time display

[1131] If the sorting is done properly, the server will award points to the user. This point information is updated in real time on the user's device, and a message such as "5 points awarded" is displayed. These points can be used for actual payments by linking with an electronic payment service.

[1132] 5. Environmental education content and point linkage

[1133] Users can also view environmental education content, including videos and articles about recycling. The server assigns points based on the viewing information and sends the information to the user's device. For example, the server displays, "You have watched a video about recycling and received 10 points."

[1134] 6. Question Answering Using Natural Language Processing

[1135] When a user enters a question about sorting, the question is sent to the server. The server analyzes the question using a natural language processing algorithm, generates an appropriate answer, and sends it to the user's device. The answer displayed is, "Please separate the caps as burnable waste."

[1136] Specific examples

[1137] Photo and token example:

[1138] The user takes a photo of the plastic bottle after drinking and sends it through the app. The server identifies it as a plastic bottle. The user is given 5 points along with the instruction to "separate the plastic bottle as plastic waste."

[1139] Example prompt sentence:

[1140] "You take a photo of the waste and submit it. The server identifies it as a plastic bottle. It shows you how to sort it properly and awards you 5 points."

[1141] This system allows users to earn points by properly separating waste and use these points for electronic payment services, making it easier to practice environmental conservation activities in everyday life.

[1142] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1143] Step 1:

[1144] The user takes an image of the waste. The input is the image data taken by the user, and the output is that the image is saved on the device. The user takes a photo of the waste using the camera function of their smartphone or tablet. This data is saved in the device's memory as a raw image file.

[1145] Step 2:

[1146] The device sends the captured image data to the server. The input is the image data stored on the device, and the output is the image data sent to the server via the Internet. The device application uses the Internet connection to send the image data to the server's API endpoint as a POST request.

[1147] Step 3:

[1148] The server analyzes the received image data. The input is the image data sent to the server, and the output is the type of waste determined by image analysis. The server uses image analysis algorithms such as TensorFlow to process the image data and determine the type of waste.

[1149] Step 4:

[1150] The server generates a sorting method based on the classification results. The input is the type of waste identified, and the output is a text message describing the appropriate sorting method. For example, for PET bottles, the server generates the message "Please separate PET bottles as plastic waste."

[1151] Step 5:

[1152] The server sends the generated classification method to the user's terminal and displays it. The input is a text message of the generated classification method, and the output is the text message displayed on the user's terminal. The server sends this message to the terminal, and the terminal application displays the message on the screen.

[1153] Step 6:

[1154] The server awards points according to waste sorting. The input is confirmation of the sorting method, and the output is the awarded points. After confirming that the user has performed the specified sorting, the server adds the points to the user's account.

[1155] Step 7:

[1156] The server updates the point information to the user's terminal in real time. The input is the awarded point information, and the output is a text message of the updated point balance. For example, a message such as "5 points have been awarded. Your current point balance is 20 points" is displayed.

[1157] Step 8:

[1158] A user browses environmental education content. The input is the user's browsing behavior, and the output is the viewed content information. The user browses videos and articles about recycling on their device. This data is sent to the server.

[1159] Step 9:

[1160] The server checks the viewing information and awards points. The input is the viewing completion information, and the output is the awarded points. The server checks that the user has viewed the content, and adds points to the user based on that information.

[1161] Step 10:

[1162] The user enters a question about the sorting method. The input is the question entered by the user, and the output is the question data sent. The user enters the question using the chat function in the app and sends the data to the server.

[1163] Step 11:

[1164] The server answers questions using a natural language processing algorithm. The input is the question data sent by the user, and the output is the generated answer text. The server analyzes the question using natural language processing, generates an appropriate answer, and sends it to the terminal. An answer such as "Please separate the caps as burnable waste" is displayed.

[1165] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1166] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1167] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1168] [Third embodiment]

[1169] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1170] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1171] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1172] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1173] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1174] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1175] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1176] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1177] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1178] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1179] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1180] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1181] The present invention is a system that supports users in taking photos of waste and sorting it appropriately. This system is composed of components such as a terminal, a server, and a user.

[1182] System Overview

[1183] 1. Taking photos of trash and sending the data

[1184] Users take photos of waste using their devices (smartphones or tablets), and the devices send the photo data to the server. For example, if a user takes a photo of a plastic bottle, the photo data is sent to the server.

[1185] 2. Image analysis on the server

[1186] The server passes the received photo data to an image analysis algorithm, which determines the type of waste (e.g., plastic bottle, paper, metal, etc.) based on its shape and characteristics. After analysis, the server confirms that the waste is a plastic bottle.

[1187] 3. Presentation of sorting methods

[1188] Based on the analysis results, the server generates a sorting method appropriate for the PET bottles. For example, it generates a message such as "Please separate PET bottles as plastic waste." This message is sent to the terminal and displayed to the user.

[1189] 4. Real-time chat support

[1190] If the user has any further questions about the sorting method, they can use the chat function on their device to input their question. For example, they can input, "How should I dispose of this bottle cap?" The device sends this question to the server, which then analyzes the question using a natural language processing algorithm and generates an answer. The answer generated, "Please sort the cap as burnable waste," is sent to the device and displayed to the user.

[1191] 5. Environmental education and point redemption

[1192] The user views environmental education content (e.g., videos or articles about recycling). The device sends the viewing information to the server. The server confirms that the viewing has been completed and awards points to the user based on that information. The point information is updated in real time on the user's device and displayed. For example, it may say, "10 points awarded."

[1193] 6. Community Participation and Support

[1194] Users join online communities and participate in discussions and events related to environmental protection. The device notifies the server of their participation. The server records the participation information and awards rewards and badges as necessary. This information is also sent to the device and displayed to the user. For example, the server may notify the user that they have earned a recycling master badge.

[1195] Specific examples

[1196] Example 1: Taking photos of garbage and showing how to separate it

[1197] The user takes a photo of a plastic bottle with their device and sends the data to the server. The server recognizes it as a plastic bottle and generates a message saying "Please separate plastic bottles as plastic waste" and sends it to the device. The message is displayed on the user's device, allowing the user to separate the bottle correctly.

[1198] Example 2: Watching environmental education videos and earning points

[1199] The user watches a video about recycling on their device, and after completing the viewing, the information is sent to the server. The server confirms the completion of the viewing and awards the user 10 points. The user's device displays "10 points awarded," encouraging environmental education.

[1200] This is a support system that helps users easily separate their garbage and raise environmental awareness. It is a powerful tool for achieving sustainable living by providing real-time advice, reward points, and encouraging community participation.

[1201] The processing flow will be explained below.

[1202] Step 1:

[1203] The user launches the camera app on the device and takes a photo of the waste (e.g., a plastic bottle).

[1204] Step 2:

[1205] The device temporarily stores the captured photo data and prepares an HTTP POST request to send it to the server.

[1206] Step 3:

[1207] The device sends the photo data to the server via an HTTP POST request.

[1208] Step 4:

[1209] The server receives the HTTP POST request and retrieves the photo data.

[1210] Step 5:

[1211] The server uses an AI model for image analysis to analyze the photo data and determine the type of waste, using feature extraction and identification algorithms.

[1212] Step 6:

[1213] Based on the analysis results, the server generates an appropriate method for separating waste according to the type of waste. For example, it generates a message such as "Please separate PET bottles as plastic waste."

[1214] Step 7:

[1215] The server constructs a response in JSON format that includes the generated sorting method and sends it to the terminal.

[1216] Step 8:

[1217] The terminal receives and analyzes the response from the server.

[1218] Step 9:

[1219] The terminal displays the analyzed sorting method to the user.

[1220] Step 10:

[1221] If the user has any further questions about the sorting method, they can use the chat function on the device to type in their questions.

[1222] Step 11:

[1223] The terminal sends the user's question to the server.

[1224] Step 12:

[1225] The server receives the question and uses natural language processing (NLP) algorithms to analyze the question and generate an answer.

[1226] Step 13:

[1227] The server sends the generated response to the terminal.

[1228] Step 14:

[1229] The terminal receives the response and displays it to the user.

[1230] Step 15:

[1231] A user selects environmental education content (e.g., a video about recycling) on ​​the device to watch.

[1232] Step 16:

[1233] The terminal notifies the server of the user's viewing actions.

[1234] Step 17:

[1235] When the user finishes watching the video, the device sends that information to the server.

[1236] Step 18:

[1237] The server confirms that the viewing has been completed and awards points to the user.

[1238] Step 19:

[1239] The server transmits the awarded point information to the terminal.

[1240] Step 20:

[1241] The terminal displays the updated points information to the user.

[1242] Step 21:

[1243] When a user participates in a discussion or event in an online community, the terminal notifies the server of the participation information.

[1244] Step 22:

[1245] The server records participation information and awards rewards and badges to users as appropriate.

[1246] Step 23:

[1247] The server sends reward and badge information to the terminal.

[1248] Step 24:

[1249] The terminal displays reward and badge information to the user.

[1250] Example 1

[1251] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1252] In conventional waste sorting systems, users have difficulty accurately identifying the type of waste and rarely have detailed knowledge of how to sort it. Furthermore, existing systems make it difficult for users to receive ongoing environmental education or community activity support, resulting in insufficient improvement in users' environmental awareness. Therefore, there is a need for a system that promotes sustainable lifestyles by supporting proper waste sorting and providing users with ongoing environmental education and community activities.

[1253] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1254] In this invention, the server includes: a means for a user to acquire an image of waste; a means for transmitting image data from a terminal to the server; a means for the server to analyze the received image data and identify the type of waste; a means for generating a recommended sorting method based on the type of waste identified by the server; a means for transmitting the generated sorting method to the terminal and displaying it; a means for the user to input additional questions and transmit them from the terminal to the server; a means for the server to analyze the questions and generate answers using natural language processing; and a means for transmitting the generated answers to the terminal and displaying them. This allows users to easily and accurately sort waste and obtain necessary information in real time. Furthermore, the system includes a means for transmitting information about viewing environmental education content to the server and awarding points, and a means for recording participation information in the online community and awarding rewards and badges, thereby improving users' environmental awareness and supporting sustainable lifestyles.

[1255] "User" refers to an individual who uses the system to perform operations related to waste separation and environmental education.

[1256] A "terminal" is a communication device used by a user, and specifically includes a smartphone, tablet, computer, etc.

[1257] A "server" is a computing device that receives and processes data sent from a terminal, and is responsible for analyzing the data and generating results.

[1258] "Image data" refers to photographic information of waste taken by a user using a terminal.

[1259] "Image analysis" refers to the computational process in which the server processes image data to determine the type of waste.

[1260] "Sorting methods" refer to specific procedures and rules for properly classifying and disposing of waste.

[1261] "Natural language processing" refers to a technology that allows a server to understand questions or messages entered by users and generate appropriate answers.

[1262] "Environmental education content" refers to teaching materials and resources that users can view to increase their environmental knowledge and awareness.

[1263] "Points" refer to rewards given to users for viewing environmental education content or using the system.

[1264] "Online community" refers to a network group in which users can participate to exchange information and engage in activities related to environmental protection.

[1265] "Rewards" refers to incentives such as badges and points that users receive through their activities in online communities.

[1266] "Badge" refers to a digital award or recognition given to users to recognize their achievements in online communities.

[1267] The present invention is a system that supports users in separating waste materials easily and appropriately. This system is composed of elements such as terminals, servers, and users. Specifically, it operates as follows.

[1268] Taking photos of trash and sending the data

[1269] Users take photos of waste using a device (e.g., a smartphone or tablet). The captured photo data is sent from the device to a server. This transmission is performed using a REST API using the HTTP protocol.

[1270] Image analysis on the server

[1271] The server passes the received image data to an image analysis algorithm. Specific algorithms used include image classification models such as TensorFlow and OpenCV. The image analysis algorithm identifies the shape and characteristics of the waste and identifies its type (e.g., plastic bottles, paper, metal, etc.).

[1272] Presentation of sorting methods

[1273] The server generates an appropriate sorting method based on the results of image analysis. For example, if it identifies a plastic bottle, it generates a message saying, "Please separate plastic bottles as plastic waste." This message is sent from the server to the terminal and displayed to the user.

[1274] Real-time chat support

[1275] If the user has any additional questions about the sorting method, they can enter them using the chat function on their device. This question is then sent from the device to the server. The server then analyzes the received question using a natural language processing algorithm (e.g., GPT-3) and generates an appropriate answer. The generated answer is then sent to the device and displayed to the user.

[1276] Environmental education and point redemption

[1277] The user views environmental education content (for example, videos or articles about recycling) on ​​the device. The viewing information is sent from the device to the server, which then confirms that the viewing is complete and awards points to the user. For example, a message saying "10 points awarded" is displayed on the device.

[1278] Community participation and support

[1279] Users join online communities and participate in discussions and events related to environmental protection. Participation information is sent from the device to the server, which records the participation information and awards rewards and badges as necessary. For example, a message saying "You have earned the Recycling Master badge" is displayed on the device.

[1280] Specific examples

[1281] Example 1: Taking photos of garbage and showing how to separate it

[1282] A user takes a photo of a plastic bottle with their device and sends the data to the server. The server recognizes it as a plastic bottle, generates a message saying "Please separate plastic bottles as plastic waste," and sends it to the device. The message is displayed on the user's device, allowing the user to separate the bottles correctly.

[1283] Example of a generative AI model input prompt: "I took a photo of some waste. How should I sort it?"

[1284] Example 2: Watching environmental education videos and earning points

[1285] A user watches a video about recycling on their device, and after completing the viewing, the information is sent to the server. The server confirms the completion of the viewing and awards the user 10 points. The user's device displays "10 points awarded," encouraging environmental education.

[1286] Example of a generative AI model input prompt: "I watched the recycling video to the end. Please give me points."

[1287] By combining these functions, the system helps users separate waste accurately and efficiently, and increases environmental awareness. It supports the development of sustainable lifestyles through real-time advice, reward points, and community participation.

[1288] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1289] Step 1: User takes a photo on their device

[1290] Users take photos of waste using devices such as smartphones or tablets. The input for this process is the image of waste taken by the user using the device's camera function, and the output is image data. Specifically, the user launches the camera app, points it at the waste, and presses the shutter button.

[1291] Step 2: The device sends the photo data to the server

[1292] The device sends the captured photo data to the server. The input to this process is the image data stored on the device, and the output is the transfer of the image data to the server. A REST API using the HTTP protocol is used to send the data. Specifically, the application on the device generates an HTTP request containing the image data and sends it to the specified server endpoint.

[1293] Step 3: The server receives the image data.

[1294] The server receives image data sent from the terminal. The input of this process is image data sent via the network, and the output is image data saved on the server. Specifically, the server receives an HTTP request and temporarily saves the image data.

[1295] Step 4: The server performs image analysis

[1296] The server passes the image data to an image analysis algorithm. The input for this process is the image data stored on the server, and the output is an analysis result indicating the type of waste. Specifically, the image data is analyzed using an image classification model using TensorFlow and OpenCV to obtain classification results such as PET bottles, paper, and metal.

[1297] Step 5: The server generates the sorting method

[1298] The server generates a sorting method based on the results of image analysis. The input to this process is the result of image analysis, and the output is a message indicating the sorting method. Specifically, it generates a message such as "Please separate PET bottles as plastic waste" based on a predefined database and rules.

[1299] Step 6: The server sends a classification method message to the terminal.

[1300] The server sends the generated sorting method message to the terminal. The input of this process is the generated sorting method message, and the output is message delivery to the terminal. Specifically, the server sends the message to the terminal using an HTTP response or a messaging queue.

[1301] Step 7: The device displays a sorting method message to the user.

[1302] The terminal displays the received message on the sorting method to the user. The input of this process is the message on the sorting method sent from the server, and the output is the message displayed on the terminal screen. Specifically, the terminal application receives the message and displays it on the user interface (UI).

[1303] Step 8: User enters follow-up question into terminal

[1304] The user enters additional questions about the sorting method using the chat function on the device. The input for this process is the user's question text, and the output is the question data entered into the device. Specifically, the user opens a chat app and enters, "How should I dispose of this bottle cap?"

[1305] Step 9: The device sends the query data to the server

[1306] The terminal sends the entered question data to the server. The input of this process is the question data entered into the terminal, and the output is the transfer of the question data to the server. Specifically, the terminal generates an HTTP request including the question data and sends it to the server.

[1307] Step 10: The server performs natural language processing and generates an answer

[1308] The server analyzes the received question data using a natural language processing algorithm and generates an answer. The input for this process is the question data, and the output is the generated answer text. Specifically, it uses a natural language processing model such as GPT-3 to generate the answer "Please separate the caps as burnable garbage."

[1309] Step 11: The server sends the response data to the terminal.

[1310] The server sends the generated answer data to the terminal. The input of this process is the generated answer text, and the output is the transfer of the answer data to the terminal. Specifically, the server generates an HTTP response including the answer data and sends it to the terminal.

[1311] Step 12: The device displays the response data to the user

[1312] The terminal displays the received answer data to the user. The input of this process is the answer text sent from the server, and the output is the answer displayed on the terminal screen. Specifically, the terminal application receives the answer data and displays it on the user interface (UI).

[1313] In this way, the system helps users separate waste accurately and easily, provides additional information in real time, and provides a mechanism to increase users' environmental awareness through environmental education and community activities.

[1314] (Application example 1)

[1315] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1316] In recent years, the increase in the use of food delivery services has led to a significant increase in waste, including containers and packaging materials. While this has created a demand for proper waste separation, users find it difficult to find the time to do so due to their busy daily lives. Furthermore, the complicated separation method often leads to incorrect separation. Furthermore, the lack of incentives for separating waste often leads to a decline in motivation. A system is needed to solve these problems, encourage users to separate waste effectively, and raise their environmental awareness.

[1317] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1318] In this invention, the server includes means for a user to take a photo of waste, means for transmitting the photo data to the server, means for the server to analyze the photo data and determine the type of waste, means for generating a recommended sorting method based on the type of waste determined by the server, means for transmitting the sorting method generated by the server to the user's terminal and displaying it, means for the user to be awarded points according to their waste sorting and displaying the point information, and means for the user to view environmental education content and transmit the viewed information to the server. This allows users to easily learn the appropriate sorting method for food delivery waste, increasing their incentive to sort waste and improving overall environmental awareness.

[1319] "User" refers to a person who uses the system to separate waste.

[1320] "Waste" refers to unwanted items and garbage.

[1321] "Photo" refers to image data that a user takes to record the state of waste.

[1322] "Means" refers to the methods or devices that a system uses to achieve its goals.

[1323] "Server" refers to a central management device that receives and analyzes data sent by users, and generates and transmits the necessary information.

[1324] "Data" refers collectively to information processed within the system, such as photos taken by users and information generated by the system.

[1325] "Analysis" refers to the process by which the server processes the received photo data and identifies the type of waste.

[1326] "Sorting methods" refers to instructions and guidelines for properly disposing of and classifying waste.

[1327] "Terminal" refers to the device (such as a smartphone or tablet) that a user uses to access the system.

[1328] "Points" refer to rewards given to users for properly sorting waste and viewing environmental education content.

[1329] "Point information" refers to information indicating the status and history of points given to a user.

[1330] "Environmental education content" refers to educational materials such as videos and articles that help users increase their environmental awareness through learning.

[1331] This invention is a system that supports the separation of waste generated by food delivery. The system allows users to take photos of waste and send them to a server, which then presents the user with appropriate separation methods and provides a function that awards points for following the separation methods. The following describes in detail an embodiment of this invention.

[1332] 1. Hardware and Software Requirements

[1333] Hardware: Smartphones, servers

[1334] Software: OpenCV, Scikit-learn, Joblib, Natural Language Processing algorithms

[1335] 2. System Configuration

[1336] User's device: The user takes a photo of the waste using a smartphone or tablet, and the device sends the photo data to the server via the Internet.

[1337] Server: The server passes the received photo data to an image analysis algorithm to identify the type of waste. Based on the analysis results, it generates a recommended sorting method and sends it to the user's device. It also awards points based on the user's behavior and manages the points history.

[1338] Image analysis algorithm: Images are preprocessed using OpenCV, and waste type is identified using a Scikit-learn-based model that has been trained on image data of various waste materials.

[1339] Natural language processing algorithms: When users enter a question, the system analyzes the question and generates an appropriate answer, allowing users to receive real-time support on how to sort in detail.

[1340] 3. System Operation

[1341] Users take photos of waste generated during food delivery and send them to a server via the app. The server then passes the received photos through an image analysis algorithm to determine the type of waste. For example, if the waste is a plastic container, a message saying "Please separate it as plastic waste" is generated and displayed on the user's device. When users follow the sorting instructions, they are awarded points based on their actions, which can be used as incentives such as discounts on delivery.

[1342] Specific examples

[1343] 1. Taking photos of trash and showing how to separate it

[1344] The user takes a photo of the food container after delivery with their smartphone and sends it to the server via the app. The server analyzes the photo and displays a message to the user saying, "Please separate this plastic container as burnable garbage."

[1345] 2. Points Award

[1346] When users correctly separate their trash according to the provided separation method, the app's internal history is updated and they are awarded recycling points, which can be used for discounts on their next food delivery.

[1347] Prompt Sentence Examples

[1348] User Question: "Which garbage bag should I put this plastic container in?"

[1349] Prompt for generative AI model: "Please tell me the proper way to separate plastic containers."

[1350] User Question: "How do I dispose of paper bags?"

[1351] Prompt for generative AI model: "How do I dispose of paper bags?"

[1352] In this way, the present invention becomes a powerful tool to help users properly separate waste and promote environmental awareness.

[1353] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1354] Step 1:

[1355] The user takes a photo of the waste used in food delivery using their smartphone. To do this, the user launches the application and takes a photo using the camera function. The input is the photo of the waste taken. The output is that this photo data is saved on the device.

[1356] Step 2:

[1357] The device initiates network communication to send the captured photo data to the server. The input is the photo data, and the output is the photo data sent to the server. A secure communication method such as HTTPS is used to send the data.

[1358] Step 3:

[1359] The server passes the received photo data to the image analysis algorithm, which uses OpenCV to pre-process the photos. This pre-processing involves resizing and normalizing the photos to convert the image data into an analyzable format. The input is the photo data sent to the server, and the output is the pre-processed image data.

[1360] Step 4:

[1361] The server passes the preprocessed image data to a waste classification model. Here, a pre-trained model from Scikit-learn is used. The model analyzes the image data and identifies the type of waste. The input is the preprocessed image data, and the output is the analysis result. For example, it may be identified as a plastic container.

[1362] Step 5:

[1363] The server generates a recommended sorting method based on the analysis results. The input is the type of waste (analysis result), and the output is a message about the sorting method. For example, a message such as "Please sort this plastic container as plastic waste" is generated.

[1364] Step 6:

[1365] The server sends the generated message of the sorting method to the terminal. The input is the generated message of the sorting method, and the output is the message sent to the terminal. The message instructs the user on the sorting method.

[1366] Step 7:

[1367] The terminal displays the message of the sorting method received from the server to the user. The user checks the displayed message and understands the sorting method. The input is the message received from the server, and the output is the message displayed to the user.

[1368] Step 8:

[1369] The user must confirm the operation before points can be awarded for sorting waste. After the user completes the sorting task, the user sends the information to the server via their device. The input is the user's sorting task completion information, and the output is the information sent to the server.

[1370] Step 9:

[1371] The server receives the user's sorting completion information and awards points. The input is the user's sorting completion information, and the output is the points awarded to the user. The points can be used as a discount for the next delivery.

[1372] Step 10:

[1373] The server receives information about the user's browsing of environmental education content and awards points based on that information. The input is the user's browsing information of environmental education content, and the output is the point information awarded to the user.

[1374] Step 11:

[1375] When a user uses a discount or benefit, the point information is checked on the terminal. The terminal displays the user's point balance and history. The input is the user's point information, and the output is the point information displayed on the terminal.

[1376] Through the above steps, a system is realized in which users can properly separate their waste and earn points to receive a discount on their next food delivery.

[1377] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1378] This invention is a system that supports users in taking photos of waste and sorting it appropriately. This system consists of components: a terminal, a server, a user, and an emotion engine.

[1379] System Overview

[1380] 1. Taking photos of trash and sending the data

[1381] Users take photos of waste using their devices (smartphones or tablets), and the devices send the photo data to the server. For example, if a user takes a photo of a plastic bottle, the photo data is sent to the server.

[1382] 2. Image analysis on the server

[1383] The server passes the received photo data to an image analysis algorithm, which determines the type of waste (e.g., plastic bottle, paper, metal, etc.) based on its shape and characteristics. After analysis, the server confirms that the waste is a plastic bottle.

[1384] 3. Presentation of sorting methods

[1385] Based on the analysis results, the server generates a sorting method appropriate for the PET bottles. For example, it generates a message such as "Please separate PET bottles as plastic waste." This message is sent to the terminal and displayed to the user.

[1386] 4. Real-time chat support

[1387] If the user has any further questions about the sorting method, they can use the chat function on their device to input their question. For example, they can input, "How should I dispose of this bottle cap?" The device sends this question to the server, which then analyzes the question using a natural language processing algorithm and generates an answer. The answer generated, "Please sort the cap as burnable waste," is sent to the device and displayed to the user.

[1388] 5. Environmental education and point redemption

[1389] The user views environmental education content (e.g., videos or articles about recycling). The device sends the viewing information to the server. The server confirms that the viewing has been completed and awards points to the user based on that information. The point information is updated in real time on the user's device and displayed. For example, it may say, "10 points awarded."

[1390] 6. Community Participation and Support

[1391] Users join online communities and participate in discussions and events related to environmental protection. The device notifies the server of their participation. The server records the participation information and awards rewards and badges as necessary. This information is also sent to the device and displayed to the user. For example, the server may notify the user that they have earned a recycling master badge.

[1392] 7. Improving user experience by incorporating an emotion engine

[1393] The emotion engine has the ability to analyze a user's facial expressions and voice and recognize their emotions. For example, when a user takes a photo or types a question using the chat function, the facial expression and voice data are acquired. The device sends this data to the server, and the server uses the emotion engine to recognize the user's emotions.

[1394] The recognized emotion information is taken into consideration when presenting methods and answers. For example, if the user is confused, a more detailed and easy-to-understand explanation is presented. If the user is satisfied, a simple message is displayed.

[1395] Specific examples

[1396] Example 1: Taking photos of garbage and showing how to separate it

[1397] The user takes a photo of a plastic bottle with their device and sends the data to the server. The server recognizes it as a plastic bottle and generates a message saying "Please separate plastic bottles as plastic waste" and sends it to the device. The message is displayed on the user's device, allowing the user to separate the bottles correctly. The user's facial expression at this time is also recorded and sent as data to the server.

[1398] Example 2: Watching environmental education videos and earning points

[1399] The user watches a video about recycling on their device, and after watching, the information is sent to the server. The server confirms that the user has watched the video and awards the user 10 points. The user's device displays a message saying "10 points awarded," encouraging them to become more environmentally educated. The user's emotions are also recorded, and if the user's satisfaction after watching is high, this information is reflected in the next content suggestions.

[1400] This invention is a support system that helps users easily separate their garbage and raise environmental awareness. It is a powerful tool for achieving sustainable living by providing real-time advice, reward points, and encouraging community participation. Furthermore, the incorporation of an emotion engine improves the user experience and provides more effective education and support.

[1401] The processing flow will be explained below.

[1402] Step 1:

[1403] The user launches the camera app on the device and takes a photo of the waste (e.g., a plastic bottle).

[1404] Step 2:

[1405] The device temporarily stores the captured photo data, acquires the user's facial expression data, and prepares an HTTP POST request to send to the server.

[1406] Step 3:

[1407] The device sends the photo data and facial expression data to the server via an HTTP POST request.

[1408] Step 4:

[1409] The server receives the HTTP POST request and retrieves the photo data and facial expression data.

[1410] Step 5:

[1411] The server uses an AI model for image analysis to analyze the photo data and determine the type of waste, using feature extraction and identification algorithms.

[1412] Step 6:

[1413] The server uses an emotion engine to analyze the facial expression data and recognize the user's emotional state (e.g., joy, confusion, indifference, etc.).

[1414] Step 7:

[1415] Based on the analysis results, the server generates an appropriate method for separating waste according to the type of waste. The content and tone of the message are adjusted depending on the user's emotional state. For example, for a confused user, the server generates a detailed explanation such as "Please separate PET bottles as plastic waste. Please see below for detailed instructions."

[1416] Step 8:

[1417] The server constructs a response in JSON format that includes the generated sorting method and sends it to the terminal.

[1418] Step 9:

[1419] The terminal receives and analyzes the response from the server.

[1420] Step 10:

[1421] The device displays the analyzed classification method to the user, and also displays additional support messages depending on the user's emotional state.

[1422] Step 11:

[1423] If the user has any further questions about the sorting method, they can use the chat function on the device to type in their questions.

[1424] Step 12:

[1425] The terminal transmits the user's question together with the reacquired facial expression data to the server.

[1426] Step 13:

[1427] The server receives the question and facial expression data, analyzes the question using natural language processing (NLP) algorithms, and reconfirms the user's emotional state.

[1428] Step 14:

[1429] The server generates an appropriate response, again adjusting it based on the emotional state, for example adding a follow-up message such as "Please separate the caps as burnable trash. If you have any further questions, please let us know."

[1430] Step 15:

[1431] The server sends the generated response to the terminal.

[1432] Step 16:

[1433] The terminal receives the response and displays it to the user.

[1434] Step 17:

[1435] A user selects environmental education content (e.g., a video about recycling) on ​​the device to watch.

[1436] Step 18:

[1437] The terminal notifies the server of the user's selection and facial expression data.

[1438] Step 19:

[1439] When the user finishes watching the video, the terminal transmits viewing completion information and facial expression data during viewing to the server.

[1440] Step 20:

[1441] The server confirms that the viewing has been completed and awards points to the user.

[1442] Step 21:

[1443] The server takes into consideration the points awarded and the user's emotional state while watching, adjusts the next recommended content, and sends it to the terminal.

[1444] Step 22:

[1445] The device displays updated point information and recommended content tailored to the user's emotional state.

[1446] Step 23:

[1447] When a user participates in a discussion or event in an online community, the terminal notifies the server of the participation information.

[1448] Step 24:

[1449] The server records participation information and awards rewards and badges to users as appropriate.

[1450] Step 25:

[1451] The server sends reward and badge information to the terminal.

[1452] Step 26:

[1453] The device displays reward and badge information to the user, and also collects emotional data from the user during the event to reflect in the next event information.

[1454] Example 2

[1455] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1456] Currently, waste sorting is generally done manually, making it difficult for users to consistently and accurately understand and follow the sorting method. Furthermore, when users have questions about the sorting method, they often cannot immediately resolve them, preventing them from sorting accurately. Furthermore, there is a lack of support to maintain motivation for waste sorting and flexible support based on users' emotions. There are also limited incentives for viewing environmental education content, making it necessary to improve eco-consciousness.

[1457] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1458] In this invention, the server includes: a device that transmits photo data and additional data to the server; a server that passes the photo data received from the device to an image analysis algorithm to identify the type of waste; a device that generates a recommended sorting method based on the type of waste identified by the server; a device that transmits the generated sorting method to the terminal and displays it to the user; a device that acquires the user's facial expression and voice data and transmits it to the server; and a server that recognizes the user's emotions using an emotion analysis algorithm and uses that information to present the sorting method. This allows users to easily and accurately sort waste and improves the user experience by providing real-time question and answer services and flexible support based on emotions. Furthermore, a points system for viewing environmental education content can improve eco-consciousness and promote sustainable behavior.

[1459] "User" refers to an individual or organization that uses this system to separate waste.

[1460] "Terminal" refers to an electronic device such as a smartphone or tablet used by a user.

[1461] A "server" refers to a computer system that receives and analyzes data sent from a terminal and sends the results back to the terminal.

[1462] "Photo data" refers to image data of waste photographed by a user using a terminal.

[1463] "Additional data" refers to additional information such as GPS data and timestamps that are acquired along with the photo data.

[1464] "Image analysis algorithm" refers to a machine learning algorithm that analyzes received photo data and determines the type of waste.

[1465] "Waste type" refers to the category into which waste is classified, examples include plastic bottles, paper, metal, etc.

[1466] "Sorting method" refers to the appropriate disposal method recommended to the user based on the identified type of waste.

[1467] "Facial expression data" refers to data relating to the user's facial expressions.

[1468] "Voice data" refers to voice data relating to the content of a user's speech.

[1469] An "emotion analysis algorithm" refers to an algorithm that analyzes facial expression data and voice data to estimate a user's emotions.

[1470] A "natural language processing algorithm" refers to an algorithm that analyzes a user's question or input text and generates an appropriate answer.

[1471] "Environmental education content" refers to information such as videos and articles that are intended to educate people about recycling and environmental protection.

[1472] "Points" refer to rewards given to users for viewing environmental education content and engaging in proper waste sorting activities.

[1473] The system of the present invention assists users in taking photos of waste and sorting it appropriately. This system is implemented by users using devices such as smartphones or tablets, which communicate with a server. The system also incorporates an emotion engine and natural language processing algorithms to improve the user experience.

[1474] Hardware and Software

[1475] 1. Terminal

[1476] Electronic devices such as smartphones and tablets.

[1477] Includes a camera, GPS module, microphone, etc.

[1478] 2. Server

[1479] A computer system that processes and analyzes data.

[1480] Implemented database, image analysis algorithm, natural language processing algorithm, and sentiment analysis algorithm.

[1481] 3. Software

[1482] Image analysis algorithms (e.g. TensorFlow)

[1483] Natural language processing algorithms (e.g., OpenAI's GPT-3)

[1484] Sentiment analysis algorithms (e.g., Microsoft's Azure Cognitive Services)

[1485] Data processing and calculation

[1486] 1. Taking photos of trash and sending the data

[1487] The user uses the terminal to take a photo of the waste.

[1488] The device sends the photo data along with GPS data and a timestamp to the server.

[1489] 2. Image analysis on the server

[1490] The server passes the received photo data to an image analysis algorithm.

[1491] The algorithm identifies the type of waste (plastic bottles, paper, metal, etc.).

[1492] 3. Presentation of sorting methods

[1493] The server generates an appropriate sorting method based on the results of image analysis.

[1494] The server sends the generated sorting method to the terminal and displays it to the user.

[1495] 4. Real-time chat support

[1496] The user enters a question using the chat function of the device.

[1497] The server uses natural language processing algorithms to analyze the question and generate an answer.

[1498] The generated answer is sent to the terminal and displayed to the user.

[1499] 5. Environmental education and point redemption

[1500] The user views the environmental education content on the device.

[1501] The terminal transmits the browsing information to the server.

[1502] The server confirms that the viewing is complete and gives points to the user.

[1503] Points information is displayed on the device in real time.

[1504] 6. Community Participation and Support

[1505] Users participate in online communities and engage in activities.

[1506] The terminal transmits the participation information to the server.

[1507] The server will award rewards and badges based on activity records.

[1508] Reward information will be displayed on the device.

[1509] 7. Use of Emotion Engines

[1510] When the device takes a photo or uses the chat function, it collects the user's facial expressions and voice data.

[1511] The terminal transmits this data to the server.

[1512] The server analyzes the user's emotions using an emotion analysis algorithm.

[1513] Based on the analysis results, sorting methods and responses are individually adjusted to improve the user experience.

[1514] Specific examples

[1515] Example 1: Taking photos of garbage and showing how to separate it

[1516] The user takes a photo of the plastic bottle with the device and sends the data to the server.

[1517] The server recognizes it as a plastic bottle, generates a message saying "Please separate plastic bottles as plastic waste," and sends it to the terminal.

[1518] The message will be displayed on the user's terminal, allowing the user to properly distinguish between the two.

[1519] The user's facial expression at this time is also recorded and sent as data to the server.

[1520] Example prompt sentence:

[1521] Photo of a plastic bottle. How should I dispose of it?

[1522] Example 2: Watching environmental education videos and earning points

[1523] Users watch videos about recycling on their devices, and after they finish watching, the information is sent to the server.

[1524] The server confirms that the viewing has been completed and awards the user 10 points.

[1525] The user's device will display the message "10 points awarded," encouraging environmental education.

[1526] The user's emotions are also recorded at this time, and if the user is highly satisfied after viewing, this information is reflected in the next content suggestions.

[1527] Example prompt sentence:

[1528] I have completed watching the recycling video. How many points did I earn?

[1529] This invention is a support system that helps users easily separate their garbage and raise environmental awareness. It is a powerful tool for achieving sustainable living by providing real-time advice, reward points, and encouraging community participation. Furthermore, the incorporation of an emotion engine improves the user experience and provides more effective education and support.

[1530] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1531] Step 1:

[1532] Garbage photography

[1533] Operation:

[1534] Users take photos of waste using the camera function on their device, such as a smartphone or tablet.

[1535] input:

[1536] The physical object of waste (e.g., plastic bottles).

[1537] output:

[1538] Save as image data on the device.

[1539] Specific operation:

[1540] The user opens the camera app, points the lens at the waste, and presses the shutter button. The device takes a photo and saves the image data to the internal storage. GPS data and a timestamp are also acquired.

[1541] Step 2:

[1542] Sending photo data

[1543] Operation:

[1544] The device sends the photo data and additional data (GPS data, timestamp) to the server.

[1545] input:

[1546] Image data, GPS data, timestamp.

[1547] output:

[1548] Data transmission to the server completed.

[1549] Specific operation:

[1550] The user presses the "Send" button in the application on the device. The device uploads the photo data and additional data to the server via the Internet. The server receives the data and temporarily stores it in a database.

[1551] Step 3:

[1552] Receiving and storing image data

[1553] Operation:

[1554] The server receives the photo data and additional data sent from the terminal and temporarily stores them.

[1555] input:

[1556] Image data, GPS data, timestamp.

[1557] output:

[1558] Storing data in a database.

[1559] Specific operation:

[1560] When the server receives the data sent from the terminal, it temporarily stores the image data and additional data in the database, and creates a new entry in the database according to the existing data structure.

[1561] Step 4:

[1562] Starting image analysis

[1563] Operation:

[1564] The server passes the received photo data to an image analysis algorithm to determine the type of waste.

[1565] input:

[1566] Image data.

[1567] output:

[1568] Analysis results (type of waste).

[1569] Specific operation:

[1570] The server invokes an image analysis algorithm (e.g., TensorFlow) to analyze the stored image data. The algorithm extracts features and identifies the type of waste in the image. The results are returned as text, such as "PET bottle."

[1571] Step 5:

[1572] Generation of separation methods

[1573] Operation:

[1574] The server generates an appropriate classification method based on the analysis results.

[1575] input:

[1576] Analysis results (type of waste).

[1577] output:

[1578] Message on how to sort waste.

[1579] Specific operation:

[1580] The server selects a message template based on the type of waste and generates specific sorting instructions, such as "Please separate PET bottles as plastic waste." The generated message is temporarily stored.

[1581] Step 6:

[1582] Sending sorting instructions

[1583] Operation:

[1584] The server transmits the generated sorting method to the terminal.

[1585] input:

[1586] Message on how to sort waste.

[1587] output:

[1588] Message sent to device completed.

[1589] Specific operation:

[1590] The server sends a message including the classification method to the terminal. This message reaches the terminal via the Internet. The terminal notifies the user of the received message.

[1591] Step 7:

[1592] Display of sorting method

[1593] Operation:

[1594] The terminal displays the received sorting method to the user.

[1595] input:

[1596] Message on how to sort waste.

[1597] output:

[1598] Displayed on the user screen.

[1599] Specific operation:

[1600] The device will then display the received sorting instructions in a pop-up notification or in a message box within the app, showing the user the message, "Please separate PET bottles as plastic waste."

[1601] Step 8:

[1602] Entering user questions

[1603] Operation:

[1604] Users can use the chat function on their device to enter questions about sorting methods.

[1605] input:

[1606] User text input (question).

[1607] output:

[1608] Prepare to send question data from the terminal to the server.

[1609] Specific operation:

[1610] A user opens a chat application and types a question such as, "How should I dispose of this bottle cap?" Once the question is typed, the device prepares the data to be sent to the server.

[1611] Step 9:

[1612] Submitting Question Data

[1613] Operation:

[1614] The terminal transmits the entered question to the server.

[1615] input:

[1616] User text input (question).

[1617] output:

[1618] Question data has been sent to the server.

[1619] Specific operation:

[1620] The device sends the question data entered by the user to the server via the Internet. After sending, a message saying "Question has been sent" is displayed on the screen.

[1621] Step 10:

[1622] Question analysis and answer generation

[1623] Operation:

[1624] The server passes the question data to a natural language processing algorithm to generate an answer.

[1625] input:

[1626] User text input (question).

[1627] output:

[1628] Analysis result (answer).

[1629] Specific operation:

[1630] The server analyzes the question using a natural language processing algorithm (e.g., GPT-3). The algorithm understands the context and generates a specific answer, such as "Please separate the caps as burnable garbage." The generated answer is temporarily stored.

[1631] Step 11:

[1632] Submitting and viewing answers

[1633] Operation:

[1634] The server sends the generated answer to the terminal, which displays it to the user.

[1635] input:

[1636] Analysis result (answer).

[1637] output:

[1638] The response is sent to the device and displayed on the user's screen.

[1639] Specific operation:

[1640] The server sends a message containing the answer to the terminal, which displays the received answer in real time, showing the user the message "Please separate the caps as burnable garbage."

[1641] Step 12:

[1642] Viewing environmental education content and earning points

[1643] Operation:

[1644] The user watches the environmental education content, and the terminal sends the viewing completion information to the server. The server confirms the viewing completion and awards points.

[1645] input:

[1646] Viewing data for educational content.

[1647] output:

[1648] Points are awarded.

[1649] Specific operation:

[1650] The user watches an environmental education video, and after watching it, the device sends the information to the server. The server checks the viewing data and awards the user 10 points. The device then displays a notification to the user that points have been awarded.

[1651] Step 13:

[1652] Community Participation and Rewards

[1653] Operation:

[1654] When a user joins an online community, the terminal sends the participation information to the server, which records the participation information and grants rewards.

[1655] input:

[1656] Community participation information.

[1657] output:

[1658] Rewarding.

[1659] Specific operation:

[1660] Users join online communities and participate in discussions and events. The device sends this information to the server, which records the participation information and awards "recycling master badges" as necessary. The device then notifies the user of rewards.

[1661] Step 14:

[1662] Acquiring and analyzing emotion data

[1663] Operation:

[1664] The device captures the user's facial expressions and voice data and sends it to the server, which then analyzes their emotions using an emotion analysis algorithm.

[1665] input:

[1666] Facial expression data, voice data.

[1667] output:

[1668] Emotion analysis results.

[1669] Specific operation:

[1670] When a user takes a photo or posts a question via chat, the device uses a camera and microphone to capture facial and voice data. The device then sends this data to a server, which then runs an emotion analysis algorithm to identify the user's emotions. The analysis results are then reflected in the next step.

[1671] Step 15:

[1672] Regulating responses based on emotions

[1673] Operation:

[1674] Based on the results of the emotion analysis, the server individually adjusts the classification method and responses and sends them to the device.

[1675] input:

[1676] Emotion analysis results.

[1677] output:

[1678] Tailored response.

[1679] Specific operation:

[1680] The server generates a detailed and easy-to-understand explanation if the user is confused, or a concise message if the user is satisfied. The tailored response is sent to the terminal and displayed to the user.

[1681] (Application example 2)

[1682] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1683] Conventional waste sorting support systems help users sort waste accurately, but they lack the functionality to provide direct incentives for that behavior. Furthermore, there is no mechanism to further improve the accuracy of waste sorting and encourage users to continue using the system. As a result, users' interest and motivation declined, and participation in sustainable environmental protection activities did not expand as expected. Furthermore, because there was no point system linked to an electronic payment service, the benefits to users could not be directly linked to their daily lives.

[1684] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for a user to take an image of waste, means for transmitting image data to the server, means for the server to analyze the image data and determine the type of waste, means for generating a recommended sorting method based on the type of waste determined by the server, means for transmitting the sorting method generated by the server to the user's terminal and displaying it, and means for awarding points according to waste sorting and making the points usable for electronic payment services. This allows users to receive direct incentives by properly sorting waste, promoting continued use and expanding participation in sustainable environmental protection activities.

[1685] A "user's device" refers to an electronic device operated by a user, and mainly refers to a smartphone, tablet, or PC.

[1686] "Image data" refers to photographic data of waste that a user takes with a terminal and sends to a server.

[1687] "Server" refers to a computer system for receiving and analyzing image data via a network.

[1688] "Waste type" refers to the classification of what material or type the waste falls into (e.g., plastic bottles, paper, metal, etc.).

[1689] "Sorting method" refers to the appropriate disposal method or treatment procedure recommended based on the type of waste.

[1690] "Points" are incentives given to users who properly separate their waste, and refer to virtual currency or electronic money that can be used in electronic payment services.

[1691] "Electronic payment service" refers to an online payment system that enables users to pay for goods and services via the Internet.

[1692] "Environmental education content" refers to media such as videos, articles, and quizzes that provide users with knowledge and information about environmental protection and recycling.

[1693] "Natural language processing" refers to computer algorithms and technologies that analyze text data entered by users, understand its meaning, and generate appropriate answers.

[1694] This system allows users to take pictures of waste and properly separate it to earn points, which can then be used for electronic payment services. The system mainly consists of a user terminal, a server, an image analysis algorithm, a natural language processing algorithm, and an electronic payment platform.

[1695] 1. Use of User Devices

[1696] Users take photos of waste using devices such as smartphones or tablets. This photo data is sent to a server via the Internet. The user's device must have a camera function and be connected to the Internet.

[1697] 2. Data reception and analysis by the server

[1698] The server receives image data sent from the user's device and uses image analysis algorithms such as TensorFlow to determine the type of waste in the received image data. Depending on the type of waste, the server generates a recommended sorting method and sends it to the user's device.

[1699] 3. Display of recommended sorting methods

[1700] The user terminal receives the sorting instructions sent from the server and displays them. For example, a message such as "Please separate PET bottles as plastic waste" is displayed.

[1701] 4. Points allocation and real-time display

[1702] If the sorting is done properly, the server will award points to the user. This point information is updated in real time on the user's device, and a message such as "5 points awarded" is displayed. These points can be used for actual payments by linking with an electronic payment service.

[1703] 5. Environmental education content and point linkage

[1704] Users can also view environmental education content, including videos and articles about recycling. The server assigns points based on the viewing information and sends the information to the user's device. For example, the server displays, "You have watched a video about recycling and received 10 points."

[1705] 6. Question Answering Using Natural Language Processing

[1706] When a user enters a question about sorting, the question is sent to the server. The server analyzes the question using a natural language processing algorithm, generates an appropriate answer, and sends it to the user's device. The answer displayed is, "Please separate the caps as burnable waste."

[1707] Specific examples

[1708] Photo and token example:

[1709] The user takes a photo of the plastic bottle after drinking and sends it through the app. The server identifies it as a plastic bottle. The user is given 5 points along with the instruction to "separate the plastic bottle as plastic waste."

[1710] Example prompt sentence:

[1711] "You take a photo of the waste and submit it. The server identifies it as a plastic bottle. It shows you how to sort it properly and awards you 5 points."

[1712] This system allows users to earn points by properly separating waste and use these points for electronic payment services, making it easier to practice environmental conservation activities in everyday life.

[1713] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1714] Step 1:

[1715] The user takes an image of the waste. The input is the image data taken by the user, and the output is that the image is saved on the device. The user takes a photo of the waste using the camera function of their smartphone or tablet. This data is saved in the device's memory as a raw image file.

[1716] Step 2:

[1717] The device sends the captured image data to the server. The input is the image data stored on the device, and the output is the image data sent to the server via the Internet. The device application uses the Internet connection to send the image data to the server's API endpoint as a POST request.

[1718] Step 3:

[1719] The server analyzes the received image data. The input is the image data sent to the server, and the output is the type of waste determined by image analysis. The server uses image analysis algorithms such as TensorFlow to process the image data and determine the type of waste.

[1720] Step 4:

[1721] The server generates a sorting method based on the classification results. The input is the type of waste identified, and the output is a text message describing the appropriate sorting method. For example, for PET bottles, the server generates the message "Please separate PET bottles as plastic waste."

[1722] Step 5:

[1723] The server sends the generated classification method to the user's terminal and displays it. The input is a text message of the generated classification method, and the output is the text message displayed on the user's terminal. The server sends this message to the terminal, and the terminal application displays the message on the screen.

[1724] Step 6:

[1725] The server awards points according to waste sorting. The input is confirmation of the sorting method, and the output is the awarded points. After confirming that the user has performed the specified sorting, the server adds the points to the user's account.

[1726] Step 7:

[1727] The server updates the point information to the user's terminal in real time. The input is the awarded point information, and the output is a text message of the updated point balance. For example, a message such as "5 points have been awarded. Your current point balance is 20 points" is displayed.

[1728] Step 8:

[1729] A user browses environmental education content. The input is the user's browsing behavior, and the output is the viewed content information. The user browses videos and articles about recycling on their device. This data is sent to the server.

[1730] Step 9:

[1731] The server checks the viewing information and awards points. The input is the viewing completion information, and the output is the awarded points. The server checks that the user has viewed the content, and adds points to the user based on that information.

[1732] Step 10:

[1733] The user enters a question about the sorting method. The input is the question entered by the user, and the output is the question data sent. The user enters the question using the chat function in the app and sends the data to the server.

[1734] Step 11:

[1735] The server answers questions using a natural language processing algorithm. The input is the question data sent by the user, and the output is the generated answer text. The server analyzes the question using natural language processing, generates an appropriate answer, and sends it to the terminal. An answer such as "Please separate the caps as burnable waste" is displayed.

[1736] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1737] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1738] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1739] [Fourth embodiment]

[1740] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1741] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1742] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1743] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1744] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1745] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1746] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1747] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1748] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1749] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1750] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1751] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1752] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1753] The present invention is a system that supports users in taking photos of waste and sorting it appropriately. This system is composed of components such as a terminal, a server, and a user.

[1754] System Overview

[1755] 1. Taking photos of trash and sending the data

[1756] Users take photos of waste using their devices (smartphones or tablets), and the devices send the photo data to the server. For example, if a user takes a photo of a plastic bottle, the photo data is sent to the server.

[1757] 2. Image analysis on the server

[1758] The server passes the received photo data to an image analysis algorithm, which determines the type of waste (e.g., plastic bottle, paper, metal, etc.) based on its shape and characteristics. After analysis, the server confirms that the waste is a plastic bottle.

[1759] 3. Presentation of sorting methods

[1760] Based on the analysis results, the server generates a sorting method appropriate for the PET bottles. For example, it generates a message such as "Please separate PET bottles as plastic waste." This message is sent to the terminal and displayed to the user.

[1761] 4. Real-time chat support

[1762] If the user has any further questions about the sorting method, they can use the chat function on their device to input their question. For example, they can input, "How should I dispose of this bottle cap?" The device sends this question to the server, which then analyzes the question using a natural language processing algorithm and generates an answer. The answer generated, "Please sort the cap as burnable waste," is sent to the device and displayed to the user.

[1763] 5. Environmental education and point redemption

[1764] The user views environmental education content (e.g., videos or articles about recycling). The device sends the viewing information to the server. The server confirms that the viewing has been completed and awards points to the user based on that information. The point information is updated in real time on the user's device and displayed. For example, it may say, "10 points awarded."

[1765] 6. Community Participation and Support

[1766] Users join online communities and participate in discussions and events related to environmental protection. The device notifies the server of their participation. The server records the participation information and awards rewards and badges as necessary. This information is also sent to the device and displayed to the user. For example, the server may notify the user that they have earned a recycling master badge.

[1767] Specific examples

[1768] Example 1: Taking photos of garbage and showing how to separate it

[1769] The user takes a photo of a plastic bottle with their device and sends the data to the server. The server recognizes it as a plastic bottle and generates a message saying "Please separate plastic bottles as plastic waste" and sends it to the device. The message is displayed on the user's device, allowing the user to separate the bottle correctly.

[1770] Example 2: Watching environmental education videos and earning points

[1771] The user watches a video about recycling on their device, and after completing the viewing, the information is sent to the server. The server confirms the completion of the viewing and awards the user 10 points. The user's device displays "10 points awarded," encouraging environmental education.

[1772] This is a support system that helps users easily separate their garbage and raise environmental awareness. It is a powerful tool for achieving sustainable living by providing real-time advice, reward points, and encouraging community participation.

[1773] The processing flow will be explained below.

[1774] Step 1:

[1775] The user launches the camera app on the device and takes a photo of the waste (e.g., a plastic bottle).

[1776] Step 2:

[1777] The device temporarily stores the captured photo data and prepares an HTTP POST request to send it to the server.

[1778] Step 3:

[1779] The device sends the photo data to the server via an HTTP POST request.

[1780] Step 4:

[1781] The server receives the HTTP POST request and retrieves the photo data.

[1782] Step 5:

[1783] The server uses an AI model for image analysis to analyze the photo data and determine the type of waste, using feature extraction and identification algorithms.

[1784] Step 6:

[1785] Based on the analysis results, the server generates an appropriate method for separating waste according to the type of waste. For example, it generates a message such as "Please separate PET bottles as plastic waste."

[1786] Step 7:

[1787] The server constructs a response in JSON format that includes the generated sorting method and sends it to the terminal.

[1788] Step 8:

[1789] The terminal receives and analyzes the response from the server.

[1790] Step 9:

[1791] The terminal displays the analyzed sorting method to the user.

[1792] Step 10:

[1793] If the user has any further questions about the sorting method, they can use the chat function on the device to type in their questions.

[1794] Step 11:

[1795] The terminal sends the user's question to the server.

[1796] Step 12:

[1797] The server receives the question and uses natural language processing (NLP) algorithms to analyze the question and generate an answer.

[1798] Step 13:

[1799] The server sends the generated response to the terminal.

[1800] Step 14:

[1801] The terminal receives the response and displays it to the user.

[1802] Step 15:

[1803] A user selects environmental education content (e.g., a video about recycling) on ​​the device to watch.

[1804] Step 16:

[1805] The terminal notifies the server of the user's viewing actions.

[1806] Step 17:

[1807] When the user finishes watching the video, the device sends that information to the server.

[1808] Step 18:

[1809] The server confirms that the viewing has been completed and awards points to the user.

[1810] Step 19:

[1811] The server transmits the awarded point information to the terminal.

[1812] Step 20:

[1813] The terminal displays the updated points information to the user.

[1814] Step 21:

[1815] When a user participates in a discussion or event in an online community, the terminal notifies the server of the participation information.

[1816] Step 22:

[1817] The server records participation information and awards rewards and badges to users as appropriate.

[1818] Step 23:

[1819] The server sends reward and badge information to the terminal.

[1820] Step 24:

[1821] The terminal displays reward and badge information to the user.

[1822] Example 1

[1823] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1824] In conventional waste sorting systems, users have difficulty accurately identifying the type of waste and rarely have detailed knowledge of how to sort it. Furthermore, existing systems make it difficult for users to receive ongoing environmental education or community activity support, resulting in insufficient improvement in users' environmental awareness. Therefore, there is a need for a system that promotes sustainable lifestyles by supporting proper waste sorting and providing users with ongoing environmental education and community activities.

[1825] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1826] In this invention, the server includes: a means for a user to acquire an image of waste; a means for transmitting image data from a terminal to the server; a means for the server to analyze the received image data and identify the type of waste; a means for generating a recommended sorting method based on the type of waste identified by the server; a means for transmitting the generated sorting method to the terminal and displaying it; a means for the user to input additional questions and transmit them from the terminal to the server; a means for the server to analyze the questions and generate answers using natural language processing; and a means for transmitting the generated answers to the terminal and displaying them. This allows users to easily and accurately sort waste and obtain necessary information in real time. Furthermore, the system includes a means for transmitting information about viewing environmental education content to the server and awarding points, and a means for recording participation information in the online community and awarding rewards and badges, thereby improving users' environmental awareness and supporting sustainable lifestyles.

[1827] "User" refers to an individual who uses the system to perform operations related to waste separation and environmental education.

[1828] A "terminal" is a communication device used by a user, and specifically includes a smartphone, tablet, computer, etc.

[1829] A "server" is a computing device that receives and processes data sent from a terminal, and is responsible for analyzing the data and generating results.

[1830] "Image data" refers to photographic information of waste taken by a user using a terminal.

[1831] "Image analysis" refers to the computational process in which the server processes image data to determine the type of waste.

[1832] "Sorting methods" refer to specific procedures and rules for properly classifying and disposing of waste.

[1833] "Natural language processing" refers to a technology that allows a server to understand questions or messages entered by users and generate appropriate answers.

[1834] "Environmental education content" refers to teaching materials and resources that users can view to increase their environmental knowledge and awareness.

[1835] "Points" refer to rewards given to users for viewing environmental education content or using the system.

[1836] "Online community" refers to a network group in which users can participate to exchange information and engage in activities related to environmental protection.

[1837] "Rewards" refers to incentives such as badges and points that users receive through their activities in online communities.

[1838] "Badge" refers to a digital award or recognition given to users to recognize their achievements in online communities.

[1839] The present invention is a system that supports users in separating waste materials easily and appropriately. This system is composed of elements such as terminals, servers, and users. Specifically, it operates as follows.

[1840] Taking photos of trash and sending the data

[1841] Users take photos of waste using a device (e.g., a smartphone or tablet). The captured photo data is sent from the device to a server. This transmission is performed using a REST API using the HTTP protocol.

[1842] Image analysis on the server

[1843] The server passes the received image data to an image analysis algorithm. Specific algorithms used include image classification models such as TensorFlow and OpenCV. The image analysis algorithm identifies the shape and characteristics of the waste and identifies its type (e.g., plastic bottles, paper, metal, etc.).

[1844] Presentation of sorting methods

[1845] The server generates an appropriate sorting method based on the results of image analysis. For example, if it identifies a plastic bottle, it generates a message saying, "Please separate plastic bottles as plastic waste." This message is sent from the server to the terminal and displayed to the user.

[1846] Real-time chat support

[1847] If the user has any additional questions about the sorting method, they can enter them using the chat function on their device. This question is then sent from the device to the server. The server then analyzes the received question using a natural language processing algorithm (e.g., GPT-3) and generates an appropriate answer. The generated answer is then sent to the device and displayed to the user.

[1848] Environmental education and point redemption

[1849] The user views environmental education content (for example, videos or articles about recycling) on ​​the device. The viewing information is sent from the device to the server, which then confirms that the viewing is complete and awards points to the user. For example, a message saying "10 points awarded" is displayed on the device.

[1850] Community participation and support

[1851] Users join online communities and participate in discussions and events related to environmental protection. Participation information is sent from the device to the server, which records the participation information and awards rewards and badges as necessary. For example, a message saying "You have earned the Recycling Master badge" is displayed on the device.

[1852] Specific examples

[1853] Example 1: Taking photos of garbage and showing how to separate it

[1854] A user takes a photo of a plastic bottle with their device and sends the data to the server. The server recognizes it as a plastic bottle, generates a message saying "Please separate plastic bottles as plastic waste," and sends it to the device. The message is displayed on the user's device, allowing the user to separate the bottles correctly.

[1855] Example of a generative AI model input prompt: "I took a photo of some waste. How should I sort it?"

[1856] Example 2: Watching environmental education videos and earning points

[1857] A user watches a video about recycling on their device, and after completing the viewing, the information is sent to the server. The server confirms the completion of the viewing and awards the user 10 points. The user's device displays "10 points awarded," encouraging environmental education.

[1858] Example of a generative AI model input prompt: "I watched the recycling video to the end. Please give me points."

[1859] By combining these functions, the system helps users separate waste accurately and efficiently, and increases environmental awareness. It supports the development of sustainable lifestyles through real-time advice, reward points, and community participation.

[1860] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1861] Step 1: User takes a photo on their device

[1862] Users take photos of waste using devices such as smartphones or tablets. The input for this process is the image of waste taken by the user using the device's camera function, and the output is image data. Specifically, the user launches the camera app, points it at the waste, and presses the shutter button.

[1863] Step 2: The device sends the photo data to the server

[1864] The device sends the captured photo data to the server. The input to this process is the image data stored on the device, and the output is the transfer of the image data to the server. A REST API using the HTTP protocol is used to send the data. Specifically, the application on the device generates an HTTP request containing the image data and sends it to the specified server endpoint.

[1865] Step 3: The server receives the image data.

[1866] The server receives image data sent from the terminal. The input of this process is image data sent via the network, and the output is image data saved on the server. Specifically, the server receives an HTTP request and temporarily saves the image data.

[1867] Step 4: The server performs image analysis

[1868] The server passes the image data to an image analysis algorithm. The input for this process is the image data stored on the server, and the output is an analysis result indicating the type of waste. Specifically, the image data is analyzed using an image classification model using TensorFlow and OpenCV to obtain classification results such as PET bottles, paper, and metal.

[1869] Step 5: The server generates the sorting method

[1870] The server generates a sorting method based on the results of image analysis. The input to this process is the result of image analysis, and the output is a message indicating the sorting method. Specifically, it generates a message such as "Please separate PET bottles as plastic waste" based on a predefined database and rules.

[1871] Step 6: The server sends a classification method message to the terminal.

[1872] The server sends the generated sorting method message to the terminal. The input of this process is the generated sorting method message, and the output is message delivery to the terminal. Specifically, the server sends the message to the terminal using an HTTP response or a messaging queue.

[1873] Step 7: The device displays a sorting method message to the user.

[1874] The terminal displays the received message on the sorting method to the user. The input of this process is the message on the sorting method sent from the server, and the output is the message displayed on the terminal screen. Specifically, the terminal application receives the message and displays it on the user interface (UI).

[1875] Step 8: User enters follow-up question into terminal

[1876] The user enters additional questions about the sorting method using the chat function on the device. The input for this process is the user's question text, and the output is the question data entered into the device. Specifically, the user opens a chat app and enters, "How should I dispose of this bottle cap?"

[1877] Step 9: The device sends the query data to the server

[1878] The terminal sends the entered question data to the server. The input of this process is the question data entered into the terminal, and the output is the transfer of the question data to the server. Specifically, the terminal generates an HTTP request including the question data and sends it to the server.

[1879] Step 10: The server performs natural language processing and generates an answer

[1880] The server analyzes the received question data using a natural language processing algorithm and generates an answer. The input for this process is the question data, and the output is the generated answer text. Specifically, it uses a natural language processing model such as GPT-3 to generate the answer "Please separate the caps as burnable garbage."

[1881] Step 11: The server sends the response data to the terminal.

[1882] The server sends the generated answer data to the terminal. The input of this process is the generated answer text, and the output is the transfer of the answer data to the terminal. Specifically, the server generates an HTTP response including the answer data and sends it to the terminal.

[1883] Step 12: The device displays the response data to the user

[1884] The terminal displays the received answer data to the user. The input of this process is the answer text sent from the server, and the output is the answer displayed on the terminal screen. Specifically, the terminal application receives the answer data and displays it on the user interface (UI).

[1885] In this way, the system helps users separate waste accurately and easily, provides additional information in real time, and provides a mechanism to increase users' environmental awareness through environmental education and community activities.

[1886] (Application example 1)

[1887] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1888] In recent years, the increase in the use of food delivery services has led to a significant increase in waste, including containers and packaging materials. While this has created a demand for proper waste separation, users find it difficult to find the time to do so due to their busy daily lives. Furthermore, the complicated separation method often leads to incorrect separation. Furthermore, the lack of incentives for separating waste often leads to a decline in motivation. A system is needed to solve these problems, encourage users to separate waste effectively, and raise their environmental awareness.

[1889] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1890] In this invention, the server includes means for a user to take a photo of waste, means for transmitting the photo data to the server, means for the server to analyze the photo data and determine the type of waste, means for generating a recommended sorting method based on the type of waste determined by the server, means for transmitting the sorting method generated by the server to the user's terminal and displaying it, means for the user to be awarded points according to their waste sorting and displaying the point information, and means for the user to view environmental education content and transmit the viewed information to the server. This allows users to easily learn the appropriate sorting method for food delivery waste, increasing their incentive to sort waste and improving overall environmental awareness.

[1891] "User" refers to a person who uses the system to separate waste.

[1892] "Waste" refers to unwanted items and garbage.

[1893] "Photo" refers to image data that a user takes to record the state of waste.

[1894] "Means" refers to the methods or devices that a system uses to achieve its goals.

[1895] "Server" refers to a central management device that receives and analyzes data sent by users, and generates and transmits the necessary information.

[1896] "Data" refers collectively to information processed within the system, such as photos taken by users and information generated by the system.

[1897] "Analysis" refers to the process by which the server processes the received photo data and identifies the type of waste.

[1898] "Sorting methods" refers to instructions and guidelines for properly disposing of and classifying waste.

[1899] "Terminal" refers to the device (such as a smartphone or tablet) that a user uses to access the system.

[1900] "Points" refer to rewards given to users for properly sorting waste and viewing environmental education content.

[1901] "Point information" refers to information indicating the status and history of points given to a user.

[1902] "Environmental education content" refers to educational materials such as videos and articles that help users increase their environmental awareness through learning.

[1903] This invention is a system that supports the separation of waste generated by food delivery. The system allows users to take photos of waste and send them to a server, which then presents the user with appropriate separation methods and provides a function that awards points for following the separation methods. The following describes in detail an embodiment of this invention.

[1904] 1. Hardware and Software Requirements

[1905] Hardware: Smartphones, servers

[1906] Software: OpenCV, Scikit-learn, Joblib, Natural Language Processing algorithms

[1907] 2. System Configuration

[1908] User's device: The user takes a photo of the waste using a smartphone or tablet, and the device sends the photo data to the server via the Internet.

[1909] Server: The server passes the received photo data to an image analysis algorithm to identify the type of waste. Based on the analysis results, it generates a recommended sorting method and sends it to the user's device. It also awards points based on the user's behavior and manages the points history.

[1910] Image analysis algorithm: Images are preprocessed using OpenCV, and waste type is identified using a Scikit-learn-based model that has been trained on image data of various waste materials.

[1911] Natural language processing algorithms: When users enter a question, the system analyzes the question and generates an appropriate answer, allowing users to receive real-time support on how to sort in detail.

[1912] 3. System Operation

[1913] Users take photos of waste generated during food delivery and send them to a server via the app. The server then passes the received photos through an image analysis algorithm to determine the type of waste. For example, if the waste is a plastic container, a message saying "Please separate it as plastic waste" is generated and displayed on the user's device. When users follow the sorting instructions, they are awarded points based on their actions, which can be used as incentives such as discounts on delivery.

[1914] Specific examples

[1915] 1. Taking photos of trash and showing how to separate it

[1916] The user takes a photo of the food container after delivery with their smartphone and sends it to the server via the app. The server analyzes the photo and displays a message to the user saying, "Please separate this plastic container as burnable garbage."

[1917] 2. Points Award

[1918] When users correctly separate their trash according to the provided separation method, the app's internal history is updated and they are awarded recycling points, which can be used for discounts on their next food delivery.

[1919] Prompt Sentence Examples

[1920] User Question: "Which garbage bag should I put this plastic container in?"

[1921] Prompt for generative AI model: "Please tell me the proper way to separate plastic containers."

[1922] User Question: "How do I dispose of paper bags?"

[1923] Prompt for generative AI model: "How do I dispose of paper bags?"

[1924] In this way, the present invention becomes a powerful tool to help users properly separate waste and promote environmental awareness.

[1925] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1926] Step 1:

[1927] The user takes a photo of the waste used in food delivery using their smartphone. To do this, the user launches the application and takes a photo using the camera function. The input is the photo of the waste taken. The output is that this photo data is saved on the device.

[1928] Step 2:

[1929] The device initiates network communication to send the captured photo data to the server. The input is the photo data, and the output is the photo data sent to the server. A secure communication method such as HTTPS is used to send the data.

[1930] Step 3:

[1931] The server passes the received photo data to the image analysis algorithm, which uses OpenCV to pre-process the photos. This pre-processing involves resizing and normalizing the photos to convert the image data into an analyzable format. The input is the photo data sent to the server, and the output is the pre-processed image data.

[1932] Step 4:

[1933] The server passes the preprocessed image data to a waste classification model. Here, a pre-trained model from Scikit-learn is used. The model analyzes the image data and identifies the type of waste. The input is the preprocessed image data, and the output is the analysis result. For example, it may be identified as a plastic container.

[1934] Step 5:

[1935] The server generates a recommended sorting method based on the analysis results. The input is the type of waste (analysis result), and the output is a message about the sorting method. For example, a message such as "Please sort this plastic container as plastic waste" is generated.

[1936] Step 6:

[1937] The server sends the generated message of the sorting method to the terminal. The input is the generated message of the sorting method, and the output is the message sent to the terminal. The message instructs the user on the sorting method.

[1938] Step 7:

[1939] The terminal displays the message of the sorting method received from the server to the user. The user checks the displayed message and understands the sorting method. The input is the message received from the server, and the output is the message displayed to the user.

[1940] Step 8:

[1941] The user must confirm the operation before points can be awarded for sorting waste. After the user completes the sorting task, the user sends the information to the server via their device. The input is the user's sorting task completion information, and the output is the information sent to the server.

[1942] Step 9:

[1943] The server receives the user's sorting completion information and awards points. The input is the user's sorting completion information, and the output is the points awarded to the user. The points can be used as a discount for the next delivery.

[1944] Step 10:

[1945] The server receives information about the user's browsing of environmental education content and awards points based on that information. The input is the user's browsing information of environmental education content, and the output is the point information awarded to the user.

[1946] Step 11:

[1947] When a user uses a discount or benefit, the point information is checked on the terminal. The terminal displays the user's point balance and history. The input is the user's point information, and the output is the point information displayed on the terminal.

[1948] Through the above steps, a system is realized in which users can properly separate their waste and earn points to receive a discount on their next food delivery.

[1949] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1950] This invention is a system that supports users in taking photos of waste and sorting it appropriately. This system consists of components: a terminal, a server, a user, and an emotion engine.

[1951] System Overview

[1952] 1. Taking photos of trash and sending the data

[1953] Users take photos of waste using their devices (smartphones or tablets), and the devices send the photo data to the server. For example, if a user takes a photo of a plastic bottle, the photo data is sent to the server.

[1954] 2. Image analysis on the server

[1955] The server passes the received photo data to an image analysis algorithm, which determines the type of waste (e.g., plastic bottle, paper, metal, etc.) based on its shape and characteristics. After analysis, the server confirms that the waste is a plastic bottle.

[1956] 3. Presentation of sorting methods

[1957] Based on the analysis results, the server generates a sorting method appropriate for the PET bottles. For example, it generates a message such as "Please separate PET bottles as plastic waste." This message is sent to the terminal and displayed to the user.

[1958] 4. Real-time chat support

[1959] If the user has any further questions about the sorting method, they can use the chat function on their device to input their question. For example, they can input, "How should I dispose of this bottle cap?" The device sends this question to the server, which then analyzes the question using a natural language processing algorithm and generates an answer. The answer generated, "Please sort the cap as burnable waste," is sent to the device and displayed to the user.

[1960] 5. Environmental education and point redemption

[1961] The user views environmental education content (e.g., videos or articles about recycling). The device sends the viewing information to the server. The server confirms that the viewing has been completed and awards points to the user based on that information. The point information is updated in real time on the user's device and displayed. For example, it may say, "10 points awarded."

[1962] 6. Community Participation and Support

[1963] Users join online communities and participate in discussions and events related to environmental protection. The device notifies the server of their participation. The server records the participation information and awards rewards and badges as necessary. This information is also sent to the device and displayed to the user. For example, the server may notify the user that they have earned a recycling master badge.

[1964] 7. Improving user experience by incorporating an emotion engine

[1965] The emotion engine has the ability to analyze a user's facial expressions and voice and recognize their emotions. For example, when a user takes a photo or types a question using the chat function, the facial expression and voice data are acquired. The device sends this data to the server, and the server uses the emotion engine to recognize the user's emotions.

[1966] The recognized emotion information is taken into consideration when presenting methods and answers. For example, if the user is confused, a more detailed and easy-to-understand explanation is presented. If the user is satisfied, a simple message is displayed.

[1967] Specific examples

[1968] Example 1: Taking photos of garbage and showing how to separate it

[1969] The user takes a photo of a plastic bottle with their device and sends the data to the server. The server recognizes it as a plastic bottle and generates a message saying "Please separate plastic bottles as plastic waste" and sends it to the device. The message is displayed on the user's device, allowing the user to separate the bottles correctly. The user's facial expression at this time is also recorded and sent as data to the server.

[1970] Example 2: Watching environmental education videos and earning points

[1971] The user watches a video about recycling on their device, and after watching, the information is sent to the server. The server confirms that the user has watched the video and awards the user 10 points. The user's device displays a message saying "10 points awarded," encouraging them to become more environmentally educated. The user's emotions are also recorded, and if the user's satisfaction after watching is high, this information is reflected in the next content suggestions.

[1972] This invention is a support system that helps users easily separate their garbage and raise environmental awareness. It is a powerful tool for achieving sustainable living by providing real-time advice, reward points, and encouraging community participation. Furthermore, the incorporation of an emotion engine improves the user experience and provides more effective education and support.

[1973] The processing flow will be explained below.

[1974] Step 1:

[1975] The user launches the camera app on the device and takes a photo of the waste (e.g., a plastic bottle).

[1976] Step 2:

[1977] The device temporarily stores the captured photo data, acquires the user's facial expression data, and prepares an HTTP POST request to send to the server.

[1978] Step 3:

[1979] The device sends the photo data and facial expression data to the server via an HTTP POST request.

[1980] Step 4:

[1981] The server receives the HTTP POST request and retrieves the photo data and facial expression data.

[1982] Step 5:

[1983] The server uses an AI model for image analysis to analyze the photo data and determine the type of waste, using feature extraction and identification algorithms.

[1984] Step 6:

[1985] The server uses an emotion engine to analyze the facial expression data and recognize the user's emotional state (e.g., joy, confusion, indifference, etc.).

[1986] Step 7:

[1987] Based on the analysis results, the server generates an appropriate method for separating waste according to the type of waste. The content and tone of the message are adjusted depending on the user's emotional state. For example, for a confused user, the server generates a detailed explanation such as "Please separate PET bottles as plastic waste. Please see below for detailed instructions."

[1988] Step 8:

[1989] The server constructs a response in JSON format that includes the generated sorting method and sends it to the terminal.

[1990] Step 9:

[1991] The terminal receives and analyzes the response from the server.

[1992] Step 10:

[1993] The device displays the analyzed classification method to the user, and also displays additional support messages depending on the user's emotional state.

[1994] Step 11:

[1995] If the user has any further questions about the sorting method, they can use the chat function on the device to type in their questions.

[1996] Step 12:

[1997] The terminal transmits the user's question together with the reacquired facial expression data to the server.

[1998] Step 13:

[1999] The server receives the question and facial expression data, analyzes the question using natural language processing (NLP) algorithms, and reconfirms the user's emotional state.

[2000] Step 14:

[2001] The server generates an appropriate response, again adjusting it based on the emotional state, for example adding a follow-up message such as "Please separate the caps as burnable trash. If you have any further questions, please let us know."

[2002] Step 15:

[2003] The server sends the generated response to the terminal.

[2004] Step 16:

[2005] The terminal receives the response and displays it to the user.

[2006] Step 17:

[2007] A user selects environmental education content (e.g., a video about recycling) on ​​the device to watch.

[2008] Step 18:

[2009] The terminal notifies the server of the user's selection and facial expression data.

[2010] Step 19:

[2011] When the user finishes watching the video, the terminal transmits viewing completion information and facial expression data during viewing to the server.

[2012] Step 20:

[2013] The server confirms that the viewing has been completed and awards points to the user.

[2014] Step 21:

[2015] The server takes into consideration the points awarded and the user's emotional state while watching, adjusts the next recommended content, and sends it to the terminal.

[2016] Step 22:

[2017] The device displays updated point information and recommended content tailored to the user's emotional state.

[2018] Step 23:

[2019] When a user participates in a discussion or event in an online community, the terminal notifies the server of the participation information.

[2020] Step 24:

[2021] The server records participation information and awards rewards and badges to users as appropriate.

[2022] Step 25:

[2023] The server sends reward and badge information to the terminal.

[2024] Step 26:

[2025] The device displays reward and badge information to the user, and also collects emotional data from the user during the event to reflect in the next event information.

[2026] Example 2

[2027] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2028] Currently, waste sorting is generally done manually, making it difficult for users to consistently and accurately understand and follow the sorting method. Furthermore, when users have questions about the sorting method, they often cannot immediately resolve them, preventing them from sorting accurately. Furthermore, there is a lack of support to maintain motivation for waste sorting and flexible support based on users' emotions. There are also limited incentives for viewing environmental education content, making it necessary to improve eco-consciousness.

[2029] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2030] In this invention, the server includes: a device that transmits photo data and additional data to the server; a server that passes the photo data received from the device to an image analysis algorithm to identify the type of waste; a device that generates a recommended sorting method based on the type of waste identified by the server; a device that transmits the generated sorting method to the terminal and displays it to the user; a device that acquires the user's facial expression and voice data and transmits it to the server; and a server that recognizes the user's emotions using an emotion analysis algorithm and uses that information to present the sorting method. This allows users to easily and accurately sort waste and improves the user experience by providing real-time question and answer services and flexible support based on emotions. Furthermore, a points system for viewing environmental education content can improve eco-consciousness and promote sustainable behavior.

[2031] "User" refers to an individual or organization that uses this system to separate waste.

[2032] "Terminal" refers to an electronic device such as a smartphone or tablet used by a user.

[2033] A "server" refers to a computer system that receives and analyzes data sent from a terminal and sends the results back to the terminal.

[2034] "Photo data" refers to image data of waste photographed by a user using a terminal.

[2035] "Additional data" refers to additional information such as GPS data and timestamps that are acquired along with the photo data.

[2036] "Image analysis algorithm" refers to a machine learning algorithm that analyzes received photo data and determines the type of waste.

[2037] "Waste type" refers to the category into which waste is classified, examples include plastic bottles, paper, metal, etc.

[2038] "Sorting method" refers to the appropriate disposal method recommended to the user based on the identified type of waste.

[2039] "Facial expression data" refers to data relating to the user's facial expressions.

[2040] "Voice data" refers to voice data relating to the content of a user's speech.

[2041] An "emotion analysis algorithm" refers to an algorithm that analyzes facial expression data and voice data to estimate a user's emotions.

[2042] A "natural language processing algorithm" refers to an algorithm that analyzes a user's question or input text and generates an appropriate answer.

[2043] "Environmental education content" refers to information such as videos and articles that are intended to educate people about recycling and environmental protection.

[2044] "Points" refer to rewards given to users for viewing environmental education content and engaging in proper waste sorting activities.

[2045] The system of the present invention assists users in taking photos of waste and sorting it appropriately. This system is implemented by users using devices such as smartphones or tablets, which communicate with a server. The system also incorporates an emotion engine and natural language processing algorithms to improve the user experience.

[2046] Hardware and Software

[2047] 1. Terminal

[2048] Electronic devices such as smartphones and tablets.

[2049] Includes a camera, GPS module, microphone, etc.

[2050] 2. Server

[2051] A computer system that processes and analyzes data.

[2052] Implemented database, image analysis algorithm, natural language processing algorithm, and sentiment analysis algorithm.

[2053] 3. Software

[2054] Image analysis algorithms (e.g. TensorFlow)

[2055] Natural language processing algorithms (e.g., OpenAI's GPT-3)

[2056] Sentiment analysis algorithms (e.g., Microsoft's Azure Cognitive Services)

[2057] Data processing and calculation

[2058] 1. Taking photos of trash and sending the data

[2059] The user uses the terminal to take a photo of the waste.

[2060] The device sends the photo data along with GPS data and a timestamp to the server.

[2061] 2. Image analysis on the server

[2062] The server passes the received photo data to an image analysis algorithm.

[2063] The algorithm identifies the type of waste (plastic bottles, paper, metal, etc.).

[2064] 3. Presentation of sorting methods

[2065] The server generates an appropriate sorting method based on the results of image analysis.

[2066] The server sends the generated sorting method to the terminal and displays it to the user.

[2067] 4. Real-time chat support

[2068] The user enters a question using the chat function of the device.

[2069] The server uses natural language processing algorithms to analyze the question and generate an answer.

[2070] The generated answer is sent to the terminal and displayed to the user.

[2071] 5. Environmental education and point redemption

[2072] The user views the environmental education content on the device.

[2073] The terminal transmits the browsing information to the server.

[2074] The server confirms that the viewing is complete and gives points to the user.

[2075] Points information is displayed on the device in real time.

[2076] 6. Community Participation and Support

[2077] Users participate in online communities and engage in activities.

[2078] The terminal transmits the participation information to the server.

[2079] The server will award rewards and badges based on activity records.

[2080] Reward information will be displayed on the device.

[2081] 7. Use of Emotion Engines

[2082] When the device takes a photo or uses the chat function, it collects the user's facial expressions and voice data.

[2083] The terminal transmits this data to the server.

[2084] The server analyzes the user's emotions using an emotion analysis algorithm.

[2085] Based on the analysis results, sorting methods and responses are individually adjusted to improve the user experience.

[2086] Specific examples

[2087] Example 1: Taking photos of garbage and showing how to separate it

[2088] The user takes a photo of the plastic bottle with the device and sends the data to the server.

[2089] The server recognizes it as a plastic bottle, generates a message saying "Please separate plastic bottles as plastic waste," and sends it to the terminal.

[2090] The message will be displayed on the user's terminal, allowing the user to properly distinguish between the two.

[2091] The user's facial expression at this time is also recorded and sent as data to the server.

[2092] Example prompt sentence:

[2093] Photo of a plastic bottle. How should I dispose of it?

[2094] Example 2: Watching environmental education videos and earning points

[2095] Users watch videos about recycling on their devices, and after they finish watching, the information is sent to the server.

[2096] The server confirms that the viewing has been completed and awards the user 10 points.

[2097] The user's device will display the message "10 points awarded," encouraging environmental education.

[2098] The user's emotions are also recorded at this time, and if the user is highly satisfied after viewing, this information is reflected in the next content suggestions.

[2099] Example prompt sentence:

[2100] I have completed watching the recycling video. How many points did I earn?

[2101] This invention is a support system that helps users easily separate their garbage and raise environmental awareness. It is a powerful tool for achieving sustainable living by providing real-time advice, reward points, and encouraging community participation. Furthermore, the incorporation of an emotion engine improves the user experience and provides more effective education and support.

[2102] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2103] Step 1:

[2104] Garbage photography

[2105] Operation:

[2106] Users take photos of waste using the camera function on their device, such as a smartphone or tablet.

[2107] input:

[2108] The physical object of waste (e.g., plastic bottles).

[2109] output:

[2110] Save as image data on the device.

[2111] Specific operation:

[2112] The user opens the camera app, points the lens at the waste, and presses the shutter button. The device takes a photo and saves the image data to the internal storage. GPS data and a timestamp are also acquired.

[2113] Step 2:

[2114] Sending photo data

[2115] Operation:

[2116] The device sends the photo data and additional data (GPS data, timestamp) to the server.

[2117] input:

[2118] Image data, GPS data, timestamp.

[2119] output:

[2120] Data transmission to the server completed.

[2121] Specific operation:

[2122] The user presses the "Send" button in the application on the device. The device uploads the photo data and additional data to the server via the Internet. The server receives the data and temporarily stores it in a database.

[2123] Step 3:

[2124] Receiving and storing image data

[2125] Operation:

[2126] The server receives the photo data and additional data sent from the terminal and temporarily stores them.

[2127] input:

[2128] Image data, GPS data, timestamp.

[2129] output:

[2130] Storing data in a database.

[2131] Specific operation:

[2132] When the server receives the data sent from the terminal, it temporarily stores the image data and additional data in the database, and creates a new entry in the database according to the existing data structure.

[2133] Step 4:

[2134] Starting image analysis

[2135] Operation:

[2136] The server passes the received photo data to an image analysis algorithm to determine the type of waste.

[2137] input:

[2138] Image data.

[2139] output:

[2140] Analysis results (type of waste).

[2141] Specific operation:

[2142] The server invokes an image analysis algorithm (e.g., TensorFlow) to analyze the stored image data. The algorithm extracts features and identifies the type of waste in the image. The results are returned as text, such as "PET bottle."

[2143] Step 5:

[2144] Generation of separation methods

[2145] Operation:

[2146] The server generates an appropriate classification method based on the analysis results.

[2147] input:

[2148] Analysis results (type of waste).

[2149] output:

[2150] Message on how to sort waste.

[2151] Specific operation:

[2152] The server selects a message template based on the type of waste and generates specific sorting instructions, such as "Please separate PET bottles as plastic waste." The generated message is temporarily stored.

[2153] Step 6:

[2154] Sending sorting instructions

[2155] Operation:

[2156] The server transmits the generated sorting method to the terminal.

[2157] input:

[2158] Message on how to sort waste.

[2159] output:

[2160] Message sent to device completed.

[2161] Specific operation:

[2162] The server sends a message including the classification method to the terminal. This message reaches the terminal via the Internet. The terminal notifies the user of the received message.

[2163] Step 7:

[2164] Display of sorting method

[2165] Operation:

[2166] The terminal displays the received sorting method to the user.

[2167] input:

[2168] Message on how to sort waste.

[2169] output:

[2170] Displayed on the user screen.

[2171] Specific operation:

[2172] The device will then display the received sorting instructions in a pop-up notification or in a message box within the app, showing the user the message, "Please separate PET bottles as plastic waste."

[2173] Step 8:

[2174] Entering user questions

[2175] Operation:

[2176] Users can use the chat function on their device to enter questions about sorting methods.

[2177] input:

[2178] User text input (question).

[2179] output:

[2180] Prepare to send question data from the terminal to the server.

[2181] Specific operation:

[2182] A user opens a chat application and types a question such as, "How should I dispose of this bottle cap?" Once the question is typed, the device prepares the data to be sent to the server.

[2183] Step 9:

[2184] Submitting Question Data

[2185] Operation:

[2186] The terminal transmits the entered question to the server.

[2187] input:

[2188] User text input (question).

[2189] output:

[2190] Question data has been sent to the server.

[2191] Specific operation:

[2192] The device sends the question data entered by the user to the server via the Internet. After sending, a message saying "Question has been sent" is displayed on the screen.

[2193] Step 10:

[2194] Question analysis and answer generation

[2195] Operation:

[2196] The server passes the question data to a natural language processing algorithm to generate an answer.

[2197] input:

[2198] User text input (question).

[2199] output:

[2200] Analysis result (answer).

[2201] Specific operation:

[2202] The server analyzes the question using a natural language processing algorithm (e.g., GPT-3). The algorithm understands the context and generates a specific answer, such as "Please separate the caps as burnable garbage." The generated answer is temporarily stored.

[2203] Step 11:

[2204] Submitting and viewing answers

[2205] Operation:

[2206] The server sends the generated answer to the terminal, which displays it to the user.

[2207] input:

[2208] Analysis result (answer).

[2209] output:

[2210] The response is sent to the device and displayed on the user's screen.

[2211] Specific operation:

[2212] The server sends a message containing the answer to the terminal, which displays the received answer in real time, showing the user the message "Please separate the caps as burnable garbage."

[2213] Step 12:

[2214] Viewing environmental education content and earning points

[2215] Operation:

[2216] The user watches the environmental education content, and the terminal sends the viewing completion information to the server. The server confirms the viewing completion and awards points.

[2217] input:

[2218] Viewing data for educational content.

[2219] output:

[2220] Points are awarded.

[2221] Specific operation:

[2222] The user watches an environmental education video, and after watching it, the device sends the information to the server. The server checks the viewing data and awards the user 10 points. The device then displays a notification to the user that points have been awarded.

[2223] Step 13:

[2224] Community Participation and Rewards

[2225] Operation:

[2226] When a user joins an online community, the terminal sends the participation information to the server, which records the participation information and grants rewards.

[2227] input:

[2228] Community participation information.

[2229] output:

[2230] Rewarding.

[2231] Specific operation:

[2232] Users join online communities and participate in discussions and events. The device sends this information to the server, which records the participation information and awards "recycling master badges" as necessary. The device then notifies the user of rewards.

[2233] Step 14:

[2234] Acquiring and analyzing emotion data

[2235] Operation:

[2236] The device captures the user's facial expressions and voice data and sends it to the server, which then analyzes their emotions using an emotion analysis algorithm.

[2237] input:

[2238] Facial expression data, voice data.

[2239] output:

[2240] Emotion analysis results.

[2241] Specific operation:

[2242] When a user takes a photo or posts a question via chat, the device uses a camera and microphone to capture facial and voice data. The device then sends this data to a server, which then runs an emotion analysis algorithm to identify the user's emotions. The analysis results are then reflected in the next step.

[2243] Step 15:

[2244] Regulating responses based on emotions

[2245] Operation:

[2246] Based on the results of the emotion analysis, the server individually adjusts the classification method and responses and sends them to the device.

[2247] input:

[2248] Emotion analysis results.

[2249] output:

[2250] Tailored response.

[2251] Specific operation:

[2252] The server generates a detailed and easy-to-understand explanation if the user is confused, or a concise message if the user is satisfied. The tailored response is sent to the terminal and displayed to the user.

[2253] (Application example 2)

[2254] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2255] Conventional waste sorting support systems help users sort waste accurately, but they lack the functionality to provide direct incentives for that behavior. Furthermore, there is no mechanism to further improve the accuracy of waste sorting and encourage users to continue using the system. As a result, users' interest and motivation declined, and participation in sustainable environmental protection activities did not expand as expected. Furthermore, because there was no point system linked to an electronic payment service, the benefits to users could not be directly linked to their daily lives.

[2256] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for a user to take an image of waste, means for transmitting image data to the server, means for the server to analyze the image data and determine the type of waste, means for generating a recommended sorting method based on the type of waste determined by the server, means for transmitting the sorting method generated by the server to the user's terminal and displaying it, and means for awarding points according to waste sorting and making the points usable for electronic payment services. This allows users to receive direct incentives by properly sorting waste, promoting continued use and expanding participation in sustainable environmental protection activities.

[2257] A "user's device" refers to an electronic device operated by a user, and mainly refers to a smartphone, tablet, or PC.

[2258] "Image data" refers to photographic data of waste that a user takes with a terminal and sends to a server.

[2259] "Server" refers to a computer system for receiving and analyzing image data via a network.

[2260] "Waste type" refers to the classification of what material or type the waste falls into (e.g., plastic bottles, paper, metal, etc.).

[2261] "Sorting method" refers to the appropriate disposal method or treatment procedure recommended based on the type of waste.

[2262] "Points" are incentives given to users who properly separate their waste, and refer to virtual currency or electronic money that can be used in electronic payment services.

[2263] "Electronic payment service" refers to an online payment system that enables users to pay for goods and services via the Internet.

[2264] "Environmental education content" refers to media such as videos, articles, and quizzes that provide users with knowledge and information about environmental protection and recycling.

[2265] "Natural language processing" refers to computer algorithms and technologies that analyze text data entered by users, understand its meaning, and generate appropriate answers.

[2266] This system allows users to take pictures of waste and properly separate it to earn points, which can then be used for electronic payment services. The system mainly...

Claims

1. means for a user to take a photograph of the waste; means for transmitting photo data to a server; A means for the server to analyze the photo data and determine the type of waste; means for generating a recommended separation method based on the type of waste determined by the server; The system includes a means for transmitting the sorting method generated by the server to the user's terminal and displaying it.

2. 10. The system of claim 1, further comprising means for a user to ask a question about a type of waste in real time, and for the server to generate an answer to the question using natural language processing.

3. A means for a user to view environmental education content and transmit the viewing information to a server; A means for the server to confirm completion of browsing and to award points to the user based on that information; The system of claim 1 further comprising means for displaying point information to a user.

4. A means for a user to participate in activities and events of an online community and notify the server of the participation information; A means for the server to award rewards and badges to users based on the participation information; 10. The system of claim 1, further comprising means for displaying reward and badge information to the user.

Images