system

Abstract

We provide the system. [Solution] A means for acquiring images of waste via a camera, A means of determining the type of waste by using identification technology based on acquired images, A means of presenting a method for classifying waste based on the judgment results, A means of identifying and providing information on collection businesses that are suitable for the type of waste, A means of displaying to users the environmental contribution of waste recycling as numerical information, A means of conducting analysis to optimize waste management at the regional level to which each household or facility belongs, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Currently, due to improper sorting of waste, waste of resources and burden on the environment have become serious problems. In order for individuals and enterprises to effectively carry out recycling activities, accurate classification of waste and presentation of appropriate reuse methods are indispensable. However, many users do not understand the sorting rules and it is difficult to obtain correct knowledge, so recycling is often not carried out efficiently. Therefore, there is a need for a system that facilitates the process from waste sorting to recycling and enables users to easily grasp their environmental contribution.

Means for Solving the Problems

[0005] This invention provides means for acquiring images of waste via an image acquisition device, and means for determining the type of waste using an image recognition algorithm based on the acquired images. Furthermore, it includes means for immediately presenting waste sorting methods based on the determination results, and means for identifying resource recovery companies suitable for the type of waste and presenting the information to the user. In addition, it includes means for displaying the environmental contribution of waste reuse as numerical information to the user, thereby supporting correct recycling activities and improving motivation. Moreover, by adding means for natural language question analysis and answering, it is possible to quickly resolve questions that the user may face.

[0006] An "image acquisition device" is a device that acquires images of waste materials taken by the user and transmits them to the processing system.

[0007] An "image recognition algorithm" is a computational method that analyzes acquired images and identifies the type of waste contained within them.

[0008] "Waste type" refers to the classification of the material or category to which the waste belongs, and is information necessary to identify the recycling method.

[0009] "Waste sorting methods" are guidelines that show the correct disposal process for identified waste.

[0010] A "resource recovery company" is a specialized company that collects specific types of waste and reuses or disposes of them.

[0011] "Environmental contribution" is a numerical representation of the positive impact that users' recycling activities have on the environment.

[0012] "Natural language question analysis" is the process of converting user-entered questions into a form that a computer can understand and extracting relevant information.

[0013] A "user" is an individual or legal entity that uses this system to sort and recycle waste. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0015] Next, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] To implement this invention, the process begins with the user installing a dedicated application on their device. The user uses this application to take an image of the waste and send it to the server. On the server, an image recognition algorithm operates based on the received image to identify the type of waste. This algorithm can, for example, use a machine learning model to analyze features in the image and classify them into categories such as plastic, metal, or glass.

[0036] Based on the identified waste data, the server provides the user with instructions on how to properly sort the waste. This includes indicating the specific type of waste bin and disposal method required. For example, if PET bottles are identified, the server will instruct the user to "put them in the plastic recycling bin."

[0037] Furthermore, if the waste meets certain conditions, the server will provide the user with information on resource recovery companies specializing in waste collection. This includes using the user's location information to select the nearest company and present connection information. For example, if large electronic equipment is identified as waste, a list of companies that collect it can be displayed.

[0038] Users can also ask questions about waste recycling in natural language through the application. The server analyzes these questions and provides relevant information. For example, in response to the question, "Where should I dispose of cardboard?", it will provide a specific answer such as, "Put it in the paper product collection box."

[0039] The server also analyzes the user's recycling history and displays their environmental contribution numerically. This quantified environmental contribution helps maintain motivation for recycling and encourages sustainable environmental behavior. For example, it can show how much CO2 was reduced from the amount of plastic products recycled during a specific period.

[0040] Thus, the present invention realizes an interactive recycling support system that enables users to properly sort waste and easily carry out recycling activities.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The user launches a dedicated application and takes a picture of the waste using their device's camera. The captured image is then converted to the optimal format through the application.

[0044] Step 2:

[0045] The device sends the captured image data to the server. At the same time, metadata such as the user's current location and the date and time the image was taken is also sent.

[0046] Step 3:

[0047] The server passes the received image data to an image recognition algorithm to determine the type of waste. The algorithm uses a pre-trained model to extract features from the image and identify the material and category.

[0048] Step 4:

[0049] Based on the identification result, the server retrieves appropriate sorting instructions from the database and generates a message to inform the user. For example, it might create a message such as, "This waste is plastic and should be placed in the recycling bin."

[0050] Step 5:

[0051] The server further identifies a suitable resource recovery company based on the user's current location information if the waste meets certain criteria (e.g., bulky waste or special waste) and provides that information to the user.

[0052] Step 6:

[0053] When a user enters a question, the device sends it to the server. The server uses natural language processing to analyze the question, retrieve relevant information from its database, and generate an answer.

[0054] Step 7:

[0055] The server records users' past recycling activities and calculates their cumulative environmental contribution. By showing users these results in concrete numbers, it aims to improve their recycling awareness.

[0056] Step 8:

[0057] The terminal receives all responses from the server and displays them in an intuitive and easy-to-understand interface for the user. Based on this, the user can properly dispose of the waste.

[0058] (Example 1)

[0059] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0060] Conventional waste disposal systems have struggled to accurately classify materials by type and to provide users with timely and accurate information on appropriate sorting methods and collection companies. Furthermore, there has been a lack of mechanisms to quantitatively demonstrate the environmental contribution of users' recycling activities and to improve users' recycling awareness.

[0061] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0062] In this invention, the server includes means for acquiring images of materials via image acquisition means, means for classifying the type of material using a learning model based on the acquired images, and means for presenting a method for separating the materials based on the classification results. This makes it possible to automatically and accurately classify the type of material and quickly provide the user with appropriate separation methods and information on recycling companies.

[0063] "Image acquisition means" refers to a device or technology for collecting images of materials.

[0064] A "learning model" refers to an algorithm that learns patterns from data and makes predictions and classifications based on new data.

[0065] "Means of classification" refers to techniques for analyzing acquired image data and dividing it into specific categories.

[0066] "Means of providing sorting methods" refers to a function that informs the user of the appropriate processing method and location for each type of material based on its classification.

[0067] A "recycling company" refers to a company or organization that collects materials and carries out appropriate reuse or disposal.

[0068] "Environmental contribution" is a numerical indicator that quantifies the degree of environmental contribution achieved through users' recycling activities.

[0069] "Natural language questions" refer to questions or inquiries about waste disposal that users ask using everyday language.

[0070] "Information provision based on behavioral patterns" refers to a method of analyzing a user's past conscious or unconscious behavioral patterns and proposing optimized information.

[0071] This invention utilizes a dedicated application installed on a user's device. The device uses its camera to capture images of waste and transmits these images to a server via the application. Upon receiving the images, the server uses a learning model to classify the materials within the images. This learning model, for example, employs a machine learning algorithm to analyze image features and classify materials into categories such as plastic, metal, and glass.

[0072] Once classification is complete, the server presents the user with sorting instructions based on the classification results. This information is displayed on the terminal's application screen, making it easy for the user to know which recycling box to place the materials in. Furthermore, the server also provides information on appropriate recycling companies depending on the classified materials. This feature is particularly useful when the materials to be discarded meet certain conditions, and it is possible to display the nearest company using the user's location information.

[0073] Users can also ask questions using natural language through the application on their device. When a user enters a question about how to handle materials, the server analyzes the question and provides relevant information using a generative AI model. For example, in response to the question, "Where should I dispose of cardboard?", it will provide a specific answer such as, "Put it in the paper product recycling box."

[0074] Furthermore, the server can record and analyze the user's past recycling activities as a history, quantifying their environmental contribution and displaying it to the user. Quantifying environmental contribution is important for encouraging sustainable behavior among users. For example, it can display how much CO2 was reduced based on the amount of plastic recycled during a certain period.

[0075] For example, if a user takes a picture of an old newspaper and sends it to the server from the application, the server will identify it as a paper product and provide the user with instructions to "put it in the paper product collection box." Through this process, users can manage their waste more efficiently and effectively.

[0076] Example of a prompt:

[0077] "Please describe the procedure for classifying waste materials as plastic or metal based on images taken by the user."

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] The user launches a dedicated application on their device and takes a picture of the waste. The input is image data of the waste captured by the device's camera. After taking the picture, the user crops or adjusts the image as needed on the confirmation screen and presses the send button. The output is the prepared image data.

[0081] Step 2:

[0082] The terminal uploads image data sent by the user to the server. The input is the image data after it has been confirmed by the user. The terminal securely transfers this data to the server via an internet connection. The output is the image data received by the server.

[0083] Step 3:

[0084] The server uses a generative AI model to classify the materials within the received image data. The input is the received image data. The server extracts features using an image processing algorithm and inputs these features into the classification model. The model classifies the materials into categories such as plastic and metal, and the output is the classification result.

[0085] Step 4:

[0086] The server presents the user with an appropriate sorting method based on the classification results. The input is the classification result of the materials obtained by the server. The server retrieves sorting methods corresponding to the classified categories from the database and formats them as data for display on the user's terminal. The output is the sorting instruction information sent to the terminal.

[0087] Step 5:

[0088] The server provides information on appropriate waste collection companies based on specific criteria. The inputs are the classification results and the user's location information. The server searches its database of waste collection companies for the relevant waste and selects the most suitable company. The output is the company information displayed on the user's terminal.

[0089] Step 6:

[0090] The user sends recycling-related questions to the server using natural language via the application. The input is the user's question text. The terminal sends this text directly to the server. The output is the question data that reached the server.

[0091] Step 7:

[0092] The server analyzes user questions and provides relevant information using a generated AI model. The input is the user's question text. The server uses natural language processing techniques to analyze the question and generate an appropriate answer. The output is the specific answer information sent to the user.

[0093] Step 8:

[0094] The server analyzes the user's recycling activity history and presents a numerical representation of their environmental contribution. The input is past activity history data. The server retrieves the user's activity records from the database and calculates a quantitative environmental contribution. The output is the numerical environmental contribution displayed on the terminal.

[0095] (Application Example 1)

[0096] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0097] In modern urban life, proper waste classification and disposal are crucial for reducing environmental impact. However, many users lack knowledge of the complex types of waste and their disposal methods, often resulting in incorrect disposal practices. This leads to reduced recycling efficiency, resource waste, and increased environmental burden. Furthermore, collection companies also face challenges in developing collection plans based on accurate waste information, hindering the optimization of resource recovery.

[0098] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0099] In this invention, the server includes means for acquiring images of waste via a camera, means for determining the type of waste using identification technology, and means for performing analysis to optimize waste management on a regional basis. This enables users to easily and accurately classify waste and dispose of it effectively, and enables collection operators to create efficient collection plans.

[0100] A "photography device" is an electronic device used to acquire images of waste, and is equipped with a camera or sensors.

[0101] "Identification technology" refers to techniques that utilize machine learning and artificial intelligence to analyze features within images and accurately determine the type of waste.

[0102] "Classification method" refers to the procedure that specifies the appropriate disposal method according to the type of waste, in order to properly process it.

[0103] A "waste collection business" is an organization that specializes in the collection and disposal of waste, and supports the reuse of resources for the benefit of citizens.

[0104] "Environmental contribution" is an indicator that shows, in concrete numerical terms, how much a waste recycling project has contributed to environmental protection.

[0105] "Regional waste management" refers to a management method aimed at optimizing and improving the efficiency of waste generation, collection, and processing activities within a specific region.

[0106] "Analysis" is the process of scrutinizing information based on data and drawing specific conclusions or recommendations.

[0107] To implement this invention, the user must first install a dedicated application on their device. The device uses its camera function to photograph the waste and sends the image to a cloud server. On the server, an image recognition model operates using a machine learning platform such as TENSORFLOW®. This identifies the type of waste and determines the classification method. The server returns the classification results and information on nearby waste collection companies to the user's device via an API using Flask or FastAPI.

[0108] Through this application, users can ask questions using natural language via voice or text. The server then interprets these questions using natural language processing services such as Dialogflow and provides relevant information. This information includes details on proper waste disposal methods and local recycling stations.

[0109] Furthermore, the server aggregates waste data generated by multiple users and performs analysis to optimize waste management in specific areas. This analysis includes planning efficient collection routes and making suggestions aimed at improving recycling rates.

[0110] As a concrete example, a user takes a photo of unwanted paper products at home with their smartphone and sends it via the application. The server then identifies "paper / cardboard" as the classification result and returns specific instructions to the user, such as "put it in the paper product recycling box." Information on local recycling stations is also provided.

[0111] Examples of prompts to input into a generative AI model:

[0112] "Could you tell me what category this waste belongs to?"

[0113] "Where should I dispose of this waste?"

[0114] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0115] Step 1:

[0116] The user takes an image of the waste using the device's camera function. The input is the actual visual information of the waste, and the output is an image file. This image file serves as the basic data for subsequent identification processing.

[0117] Step 2:

[0118] The device sends the captured image file to the cloud server. The input is an image file, and the output is an upload to the server via the network. The image is transmitted to the server via a secure protocol.

[0119] Step 3:

[0120] The server receives the image file and uses TensorFlow to perform calculations to identify the type of waste in the image. The input is image data, and the output is the waste classification result. In this process, a pre-trained model is used to analyze the features in the image and determine which category the waste belongs to.

[0121] Step 4:

[0122] The server extracts information on appropriate disposal methods and the nearest waste collection service providers based on the classification results of the identified waste. The input is the classification result, and the output is the disposal method and service provider information. This data is retrieved by referencing information stored in a database.

[0123] Step 5:

[0124] Users can input questions about waste in natural language via a terminal. The input is natural language text, which the terminal sends to the server over the network. The output is the user's query data sent to the server.

[0125] Step 6:

[0126] The server uses Dialogflow to interpret natural language questions from the user. The input is natural language text data, and the output is the interpretation result. The server analyzes the user's intent and collects appropriate information from the database.

[0127] Step 7:

[0128] The server provides specific information to the user based on the analyzed results. The input is information extracted based on interpretation, and the output is the response information to the user. The information is sent to the device and displayed on the user's app screen.

[0129] Step 8:

[0130] The server aggregates waste data collected from all users and performs analysis to improve the efficiency of waste management at the regional level. The input is the aggregated waste data, and the output is the data analysis results. These results enable suggestions for optimizing collection routes and improving recycling rates. It also generates example prompts for inputting data into the next generation AI model.

[0131] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0132] This invention is implemented through a dedicated application installed on a user's device. The user uses this application to take images of waste. The device sends the acquired images to a server, which includes the user's current location information and related metadata. The server uses an image recognition algorithm to classify the waste and suggests sorting methods based on the results.

[0133] Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions. By analyzing the user's voice and facial expressions via the device's camera and microphone, it estimates emotions from input data and the user's actions during operation. This engine can determine whether the user is feeling stressed or highly motivated.

[0134] Based on emotional data obtained from the emotion engine, the server provides information and feedback tailored to the user. For example, if a user is dissatisfied with recycling activities, the server will generate more detailed instructions or encouraging messages and display them to the user.

[0135] Furthermore, when a user enters a text question, the device sends the information to the server, which uses natural language processing to interpret the question and generate a response that reflects the user's emotional state. For example, if a user asks, "What should I do with this material?", the server might respond, "That material is recyclable. The nearest recycling box is at XX," and may even add encouraging words.

[0136] This system also provides a means to record the user's recycling history and visualize their cumulative environmental contribution. Depending on the user's sentiment, this environmental contribution information can be provided as positive feedback, supporting the user's sustainable behavior.

[0137] This invention provides an interactive system that allows users to more effectively carry out recycling activities, understand their own contributions, and maintain their motivation.

[0138] The following describes the processing flow.

[0139] Step 1:

[0140] The user launches a dedicated application on their device and takes a picture of the waste with the camera. After taking the picture, the device sends the image data to the server. This includes metadata such as the user's location and the time the picture was taken.

[0141] Step 2:

[0142] The server processes the received images using an image recognition algorithm to identify the type of waste. The algorithm analyzes the image features and estimates categories such as plastic, glass, and metal.

[0143] Step 3:

[0144] Based on the classification results, the server searches the database for instructions on how to sort the waste and sends them to the user. For example, it might generate an instruction such as, "Put plastics in the blue recycling box."

[0145] Step 4:

[0146] If the emotion engine determines that the user is in an emotionally unstable state, the device generates additional support messages to provide the user with reassuring messages.

[0147] Step 5:

[0148] If the waste requires special handling, the server identifies the nearest appropriate resource recovery company based on the user's location and sends that information to the user.

[0149] Step 6:

[0150] When a user submits a question about recycling or waste disposal in natural language text, the device sends the question to a server. The server analyzes the question, researches relevant information, and creates a thoughtful answer tailored to the user's emotional state, which is then returned to the user.

[0151] Step 7:

[0152] The server records the user's past recycling activities and calculates their environmental contribution based on statistical data. This contribution information is adjusted according to the user's emotional state and displayed to the user as positive feedback from their device.

[0153] Step 8:

[0154] The terminal displays all responses from the server in a user-friendly interface, making it easier for users to select the appropriate waste disposal method. This entire process supports users in effective recycling activities.

[0155] (Example 2)

[0156] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0157] In recent years, as environmental problems have become more serious, proper waste disposal and resource recycling have become crucial. However, a challenge remains in providing sufficient support to help users accurately classify their waste and to motivate them to do so. In particular, there is a need for systems that provide interactive feedback based on user emotions and present specific information on waste collection companies.

[0158] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0159] In this invention, the server includes means for acquiring images of objects via an image acquisition device, means for determining the type of object based on the acquired images using pattern recognition technology, and means for analyzing the user's emotions and generating feedback corresponding to their emotional state. This enables the user to correctly classify waste, understand their environmental contribution, and receive appropriate feedback tailored to their emotions.

[0160] An "image acquisition device" is a device used to acquire images of objects, and primarily refers to a camera.

[0161] "Pattern recognition technology" is a method of analyzing the characteristics of an object from acquired image data and classifying it into known categories.

[0162] "Identifying the type of object" means using pattern recognition technology to recognize the category of a specific object from an image.

[0163] "Emotional analysis" is the process of evaluating a user's emotional state based on their voice and facial expression data.

[0164] "Feedback tailored to emotional state" refers to individually adjusted information and messages provided based on the results of the user's emotional analysis.

[0165] "Providing information on collection companies" means providing users with information about appropriate collection companies based on the type of object identified.

[0166] "Environmental contribution" is a numerical indicator that shows the extent to which the reuse and sorting activities carried out by users contribute to environmental protection.

[0167] This invention is implemented using a dedicated application installed on a user's device. The user starts the system by taking an image of the waste using the device's camera. Simultaneously, the device uses its GPS function to acquire the user's location information and transmits it to the server as metadata along with the image.

[0168] The server uses advanced pattern recognition technology and image recognition algorithms to determine the type of waste from received images. These algorithms include common deep learning models such as ResNet. Based on the classification results, the server provides the user with information on the most suitable sorting method and collection companies via the terminal.

[0169] Furthermore, the device uses a camera and microphone to collect the user's facial expressions and voice, and an emotion engine analyzes the user's emotions. This process utilizes software such as OpenFace and Vokaturi to generate and provide feedback tailored to the user's emotional state.

[0170] For example, if a user takes a picture of a plastic bottle, the server will provide information such as, "This plastic bottle can be recycled. The nearest recycling center is XX." Furthermore, if a user feels that recycling has become a hassle, the emotion engine will evaluate this state and the server will generate an encouraging message such as, "The cumulative effect of recycling activities makes a big difference, let's keep going!"

[0171] Examples of prompts used as input to a generative AI model include: "Please describe the procedure for classifying waste images taken by the user and providing specific sorting methods," and "Please tell me how to generate feedback that responds to the user's emotional state."

[0172] In this way, users can efficiently support environmental protection activities in an interactive and personalized manner.

[0173] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0174] Step 1:

[0175] The user takes an image of the waste using the device. The input is the image taken by the user. The device acquires a high-resolution image using its built-in camera and simultaneously acquires the user's location information using its GPS function. This data is stored as metadata in preparation for subsequent processing. The output is the image data of the waste and the location information.

[0176] Step 2:

[0177] The terminal transmits image data and location information of the waste to the server. Inputs include captured image data and attached location information. The terminal transmits this data to the server via the network and checks its integrity. Outputs are the transmitted image data and location information.

[0178] Step 3:

[0179] The server analyzes the received image data and uses an object recognition algorithm to determine the type of waste. The input is image data sent from the terminal. The server utilizes deep learning models such as ResNet to extract object features from the image and classify them into known categories. The output is information about the type of waste.

[0180] Step 4:

[0181] The server generates appropriate sorting methods and collection company information based on the type of waste and sends it to the terminal. The input is the identified type of waste. The server compares this with past data to determine the optimal sorting guidance and searches for information on relevant collection companies. The output is sorting methods and collection company information customized for the user.

[0182] Step 5:

[0183] The device uses its camera and microphone to send the user's facial expressions and voice to the emotion engine in real time for emotion analysis. The input is the user's facial expressions and voice data as reactions. The device sends this data to the emotion engine, which uses a complex algorithm to estimate the user's emotional state. The output is the user's emotional state data.

[0184] Step 6:

[0185] The server generates and sends feedback messages to the terminal based on the user's emotional data. The input is emotional state data analyzed by the emotion engine. The server customizes encouraging and guiding messages according to the emotional state, creating feedback to boost the user's motivation. The output is the feedback message directed at the user.

[0186] (Application Example 2)

[0187] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0188] In recent years, urban waste management has become a significant social issue, and there is a growing need to promote recycling. However, challenges remain, such as the difficulty for ordinary citizens to understand how to properly sort waste and the difficulty in maintaining motivation. Traditional methods have lacked support tailored to the individual feelings and motivations of users, leading to a decline in willingness to participate in recycling.

[0189] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0190] In this invention, the server includes means for acquiring images of waste via image acquisition means, means for determining the type of waste using image recognition processing based on the acquired images, and means for analyzing the user's emotional state and providing encouragement or instructions based on those emotions. This makes it possible for users to easily sort their waste and to increase their motivation to participate in recycling through feedback tailored to their individual emotions.

[0191] "Image acquisition means" refers to devices and technologies used to capture images of waste.

[0192] "Image recognition processing" refers to algorithms and methods for analyzing specific patterns in acquired images to determine the type of waste.

[0193] "Emotional state analysis" refers to a technology that estimates a user's emotions at a given time based on their facial expressions, voice, etc., and provides appropriate feedback.

[0194] "Providing encouragement or guidance" refers to positive messages or specific action plans sent to users in response to their analyzed emotions.

[0195] "Numerical information on environmental contribution" refers to information that quantifies and visually displays the contribution made by users through their recycling activities.

[0196] "Recording historical data" refers to data management that saves information about past recycling activities and uses it as a reference for the future.

[0197] "City-wide statistical information" refers to information compiled and analyzed from data on recycling activities within a region, in order to understand the overall situation.

[0198] This invention is implemented by an application installed on a user's information processing device. The process begins when the user takes an image of waste and the acquired image is sent to a server. The server receives the image of the waste using an image acquisition means and determines its type through image recognition processing. A generally available artificial intelligence algorithm can be applied to the image recognition processing used in this process.

[0199] The server, based on the type of waste identified from the image, presents the user with the appropriate sorting method. In this process, the user's current location information can also be used to identify the nearest waste collection point. This allows the user to sort their waste in the correct location.

[0200] Furthermore, based on information obtained through the user's camera and microphone, the server analyzes the user's emotional state and provides encouragement and appropriate instructions tailored to the user's feelings. For example, if the server analyzes that the user is feeling stressed, it will send a message such as, "Small daily actions make a big difference to the environment!" as positive feedback regarding recycling activities.

[0201] Users' recycling activity history is stored in a database, and their environmental contribution is calculated. The server can use this historical data to compile statistical information and provide it in a ranking format to raise recycling awareness throughout the city. This allows users to compare their contributions with others and gain further motivation.

[0202] The program's natural language processing uses common natural language generation techniques to provide accurate responses to text questions from users. For example, if a user asks, "How should I dispose of this material?", the system will generate a response such as, "This material is recyclable. The nearest collection point is XX."

[0203] An example of a prompt for a generative AI model, used when performing sentiment analysis, is: "Generate positive feedback based on sentiment monitoring results. The user is recycling and their current sentiment is 'motivated'." This prompt allows the system to generate a message that matches the user's sentiment.

[0204] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0205] Step 1:

[0206] The user takes a picture of the waste using their smartphone camera. The image data is sent to the server as input. In this step, the device's location information is also sent along with the captured image.

[0207] Step 2:

[0208] The server applies an image recognition algorithm to the received image data as input. Data calculations are performed to identify the type of waste, and the recognized waste category is output.

[0209] Step 3:

[0210] The server determines the appropriate waste sorting method based on the classification results and presents the information to the user's terminal. In this step, the server also uses the user's current location information to process data to identify the nearest waste collection point and outputs the results.

[0211] Step 4:

[0212] To analyze the user's emotions, the device's camera and microphone are used to collect data on facial expressions and voice. The server analyzes the input emotion data to determine the user's emotional state.

[0213] Step 5:

[0214] The server generates feedback messages based on emotional states. It utilizes a generation AI model to output appropriate messages corresponding to the input emotional data.

[0215] Step 6:

[0216] On the terminal, users can check their recycling activity history and receive data visualizing their environmental contribution. The server calculates statistical information based on the collected historical data and provides the output as a ranking to the terminal.

[0217] Step 7:

[0218] When a user enters a text question, the server analyzes it using natural language processing technology. Based on the analyzed question, it generates an appropriate answer and sends it to the user's device. The output answer also includes supplementary information based on the user's emotional state.

[0219] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0220] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0221] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0222] [Second Embodiment]

[0223] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0224] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0225] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0226] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0227] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0228] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0229] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0230] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0231] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0232] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0233] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0234] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0235] To implement this invention, the process begins with the user installing a dedicated application on their device. The user uses this application to take an image of the waste and send it to the server. On the server, an image recognition algorithm operates based on the received image to identify the type of waste. This algorithm can, for example, use a machine learning model to analyze features in the image and classify them into categories such as plastic, metal, or glass.

[0236] Based on the identified waste data, the server provides the user with instructions on how to properly sort the waste. This includes indicating the specific type of waste bin and disposal method required. For example, if PET bottles are identified, the server will instruct the user to "put them in the plastic recycling bin."

[0237] Furthermore, if the waste meets certain conditions, the server will provide the user with information on resource recovery companies specializing in waste collection. This includes using the user's location information to select the nearest company and present connection information. For example, if large electronic equipment is identified as waste, a list of companies that collect it can be displayed.

[0238] Users can also ask questions about waste recycling in natural language through the application. The server analyzes these questions and provides relevant information. For example, in response to the question, "Where should I dispose of cardboard?", it will provide a specific answer such as, "Put it in the paper product collection box."

[0239] The server also analyzes the user's recycling history and displays their environmental contribution numerically. This quantified environmental contribution helps maintain motivation for recycling and encourages sustainable environmental behavior. For example, it can show how much CO2 was reduced from the amount of plastic products recycled during a specific period.

[0240] Thus, the present invention realizes an interactive recycling support system that enables users to properly sort waste and easily carry out recycling activities.

[0241] The following describes the processing flow.

[0242] Step 1:

[0243] The user launches a dedicated application and takes a picture of the waste using their device's camera. The captured image is then converted to the optimal format through the application.

[0244] Step 2:

[0245] The device sends the captured image data to the server. At the same time, metadata such as the user's current location and the date and time the image was taken is also sent.

[0246] Step 3:

[0247] The server passes the received image data to an image recognition algorithm to determine the type of waste. The algorithm uses a pre-trained model to extract features from the image and identify the material and category.

[0248] Step 4:

[0249] Based on the identification result, the server retrieves appropriate sorting instructions from the database and generates a message to inform the user. For example, it might create a message such as, "This waste is plastic and should be placed in the recycling bin."

[0250] Step 5:

[0251] The server further identifies a suitable resource recovery company based on the user's current location information if the waste meets certain criteria (e.g., bulky waste or special waste) and provides that information to the user.

[0252] Step 6:

[0253] When a user enters a question, the device sends it to the server. The server uses natural language processing to analyze the question, retrieve relevant information from its database, and generate an answer.

[0254] Step 7:

[0255] The server records users' past recycling activities and calculates their cumulative environmental contribution. By showing users these results in concrete numbers, it aims to improve their recycling awareness.

[0256] Step 8:

[0257] The terminal receives all responses from the server and displays them in an intuitive and easy-to-understand interface for the user. Based on this, the user can properly dispose of the waste.

[0258] (Example 1)

[0259] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0260] Conventional waste disposal systems have struggled to accurately classify materials by type and to provide users with timely and accurate information on appropriate sorting methods and collection companies. Furthermore, there has been a lack of mechanisms to quantitatively demonstrate the environmental contribution of users' recycling activities and to improve users' recycling awareness.

[0261] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0262] In this invention, the server includes means for acquiring images of materials via image acquisition means, means for classifying the type of material using a learning model based on the acquired images, and means for presenting a method for separating the materials based on the classification results. This makes it possible to automatically and accurately classify the type of material and quickly provide the user with appropriate separation methods and information on recycling companies.

[0263] "Image acquisition means" refers to a device or technology for collecting images of materials.

[0264] A "learning model" refers to an algorithm that learns patterns from data and makes predictions and classifications based on new data.

[0265] "Means of classification" refers to techniques for analyzing acquired image data and dividing it into specific categories.

[0266] "Means of providing sorting methods" refers to a function that informs the user of the appropriate processing method and location for each type of material based on its classification.

[0267] A "recycling company" refers to a company or organization that collects materials and carries out appropriate reuse or disposal.

[0268] "Environmental contribution" is a numerical indicator that quantifies the degree of environmental contribution achieved through users' recycling activities.

[0269] "Natural language questions" refer to questions or inquiries about waste disposal that users ask using everyday language.

[0270] "Information provision based on behavioral patterns" refers to a method of analyzing a user's past conscious or unconscious behavioral patterns and proposing optimized information.

[0271] This invention utilizes a dedicated application installed on a user's device. The device uses its camera to capture images of waste and transmits these images to a server via the application. Upon receiving the images, the server uses a learning model to classify the materials within the images. This learning model, for example, employs a machine learning algorithm to analyze image features and classify materials into categories such as plastic, metal, and glass.

[0272] Once classification is complete, the server presents the user with sorting instructions based on the classification results. This information is displayed on the terminal's application screen, making it easy for the user to know which recycling box to place the materials in. Furthermore, the server also provides information on appropriate recycling companies depending on the classified materials. This feature is particularly useful when the materials to be discarded meet certain conditions, and it is possible to display the nearest company using the user's location information.

[0273] Users can also ask questions using natural language through the application on their device. When a user enters a question about how to handle materials, the server analyzes the question and provides relevant information using a generative AI model. For example, in response to the question, "Where should I dispose of cardboard?", it will provide a specific answer such as, "Put it in the paper product recycling box."

[0274] Furthermore, the server can record and analyze the user's past recycling activities as a history, quantifying their environmental contribution and displaying it to the user. Quantifying environmental contribution is important for encouraging sustainable behavior among users. For example, it can display how much CO2 was reduced based on the amount of plastic recycled during a certain period.

[0275] For example, if a user takes a picture of an old newspaper and sends it to the server from the application, the server will identify it as a paper product and provide the user with instructions to "put it in the paper product collection box." Through this process, users can manage their waste more efficiently and effectively.

[0276] Example of a prompt:

[0277] "Please describe the procedure for classifying waste materials as plastic or metal based on images taken by the user."

[0278] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0279] Step 1:

[0280] The user launches the dedicated application on the terminal and takes a picture of the waste. As input, image data of the waste taken with the terminal camera is obtained. After taking the picture, the user trims or adjusts the image on the confirmation screen if necessary and presses the send button. The output is the prepared image data.

[0281] Step 2:

[0282] The terminal uploads the image data sent by the user to the server. The input is the image data after the user's confirmation. The terminal securely transfers the data to the server via an internet connection. The output is the image data received by the server.

[0283] Step 3:

[0284] Based on the received image data, the server uses a generative AI model to classify the materials in the image. The input is the received image data. The server extracts features by an image processing algorithm and inputs the features into a classification model. The model classifies the materials into categories such as plastic and metal, and the output is the classification result.

[0285] Step 4:

[0286] Based on the classification result, the server presents an appropriate sorting method to the user. The input is the classification result of the materials obtained by the server. The server retrieves a sorting method corresponding to the classified category from the database and formats it as display data for the user's terminal. The output is the sorting instruction information sent to the terminal.

[0287] Step 5:

[0288] The server provides information on appropriate waste collection companies based on specific criteria. The inputs are the classification results and the user's location information. The server searches its database of waste collection companies for the relevant waste and selects the most suitable company. The output is the company information displayed on the user's terminal.

[0289] Step 6:

[0290] The user sends recycling-related questions to the server using natural language via the application. The input is the user's question text. The terminal sends this text directly to the server. The output is the question data that reached the server.

[0291] Step 7:

[0292] The server analyzes user questions and provides relevant information using a generated AI model. The input is the user's question text. The server uses natural language processing techniques to analyze the question and generate an appropriate answer. The output is the specific answer information sent to the user.

[0293] Step 8:

[0294] The server analyzes the user's recycling activity history and presents a numerical representation of their environmental contribution. The input is past activity history data. The server retrieves the user's activity records from the database and calculates a quantitative environmental contribution. The output is the numerical environmental contribution displayed on the terminal.

[0295] (Application Example 1)

[0296] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0297] In modern urban life, proper waste classification and disposal are crucial for reducing environmental impact. However, many users lack knowledge of the complex types of waste and their disposal methods, often resulting in incorrect disposal practices. This leads to reduced recycling efficiency, resource waste, and increased environmental burden. Furthermore, collection companies also face challenges in developing collection plans based on accurate waste information, hindering the optimization of resource recovery.

[0298] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0299] In this invention, the server includes means for acquiring images of waste via a camera, means for determining the type of waste using identification technology, and means for performing analysis to optimize waste management on a regional basis. This enables users to easily and accurately classify waste and dispose of it effectively, and enables collection operators to create efficient collection plans.

[0300] A "photography device" is an electronic device used to acquire images of waste, and is equipped with a camera or sensors.

[0301] "Identification technology" refers to techniques that utilize machine learning and artificial intelligence to analyze features within images and accurately determine the type of waste.

[0302] "Classification method" refers to the procedure that specifies the appropriate disposal method according to the type of waste, in order to properly process it.

[0303] A "waste collection business" is an organization that specializes in the collection and disposal of waste, and supports the reuse of resources for the benefit of citizens.

[0304] "Environmental contribution" is an indicator that shows, in concrete numerical terms, how much a waste recycling project has contributed to environmental protection.

[0305] "Waste management at the regional level" is a management method for optimizing and enhancing the efficiency of activities related to waste generation, collection, and treatment in a specific region.

[0306] "Analysis" is a process of scrutinizing information based on data to derive specific conclusions and recommendations.

[0307] To implement this invention, first, the user needs to install a dedicated application on their terminal. The terminal uses the camera function to take pictures of waste and send the images to the cloud server. On the server, a machine learning platform such as TensorFlow is utilized for the operation of the image recognition model. This enables the identification of the type of waste and the determination of the classification method. The server returns the classification results and information about nearby collection operators to the user terminal via an API using Flask or FastAPI.

[0308] Through this application, the user can ask questions in natural language, either by voice or text. In response, the server uses a natural language processing service such as Dialogflow to interpret the questions and provide relevant information. This information includes the appropriate disposal methods for waste and details of recycling stations in each region.

[0309] Furthermore, the server aggregates waste data generated by multiple users and performs analysis to optimize waste management in a specific region. This analysis includes planning efficient collection routes and making proposals aimed at improving the recycling rate.

[0310] As a specific example, the user takes a picture of unwanted paper products at home with their smartphone and sends them via the application. Subsequently, the server identifies "paper and cardboard" as the classification result and returns a specific instruction to the user, such as "put it in the paper product recycling box". Information about the regional recycling station is also provided.

[0311] Examples of prompt sentences for input into the generative AI model:

[0312] "Could you tell me what category this waste belongs to?"

[0313] "Where should I dispose of this waste?"

[0314] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0315] Step 1:

[0316] The user takes an image of the waste using the device's camera function. The input is the actual visual information of the waste, and the output is an image file. This image file serves as the basic data for subsequent identification processing.

[0317] Step 2:

[0318] The device sends the captured image file to the cloud server. The input is an image file, and the output is an upload to the server via the network. The image is transmitted to the server via a secure protocol.

[0319] Step 3:

[0320] The server receives the image file and uses TensorFlow to perform calculations to identify the type of waste in the image. The input is image data, and the output is the waste classification result. In this process, a pre-trained model is used to analyze the features in the image and determine which category the waste belongs to.

[0321] Step 4:

[0322] The server extracts information on appropriate disposal methods and the nearest waste collection service providers based on the classification results of the identified waste. The input is the classification result, and the output is the disposal method and service provider information. This data is retrieved by referencing information stored in a database.

[0323] Step 5:

[0324] Users can input questions about waste in natural language via a terminal. The input is natural language text, which the terminal sends to the server over the network. The output is the user's query data sent to the server.

[0325] Step 6:

[0326] The server uses Dialogflow to interpret natural language questions from the user. The input is natural language text data, and the output is the interpretation result. The server analyzes the user's intent and collects appropriate information from the database.

[0327] Step 7:

[0328] The server provides specific information to the user based on the analyzed results. The input is information extracted based on interpretation, and the output is the response information to the user. The information is sent to the device and displayed on the user's app screen.

[0329] Step 8:

[0330] The server aggregates waste data collected from all users and performs analysis to improve the efficiency of waste management at the regional level. The input is the aggregated waste data, and the output is the data analysis results. These results enable suggestions for optimizing collection routes and improving recycling rates. It also generates example prompts for inputting data into the next generation AI model.

[0331] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0332] This invention is implemented through a dedicated application installed on a user's device. The user uses this application to take images of waste. The device sends the acquired images to a server, which includes the user's current location information and related metadata. The server uses an image recognition algorithm to classify the waste and suggests sorting methods based on the results.

[0333] Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions. By analyzing the user's voice and facial expressions via the device's camera and microphone, it estimates emotions from input data and the user's actions during operation. This engine can determine whether the user is feeling stressed or highly motivated.

[0334] Based on emotional data obtained from the emotion engine, the server provides information and feedback tailored to the user. For example, if a user is dissatisfied with recycling activities, the server will generate more detailed instructions or encouraging messages and display them to the user.

[0335] Furthermore, when a user enters a text question, the device sends the information to the server, which uses natural language processing to interpret the question and generate a response that reflects the user's emotional state. For example, if a user asks, "What should I do with this material?", the server might respond, "That material is recyclable. The nearest recycling box is at XX," and may even add encouraging words.

[0336] This system also provides a means to record the user's recycling history and visualize their cumulative environmental contribution. Depending on the user's sentiment, this environmental contribution information can be provided as positive feedback, supporting the user's sustainable behavior.

[0337] This invention provides an interactive system that allows users to more effectively carry out recycling activities, understand their own contributions, and maintain their motivation.

[0338] The following describes the processing flow.

[0339] Step 1:

[0340] The user launches a dedicated application on their device and takes a picture of the waste with the camera. After taking the picture, the device sends the image data to the server. This includes metadata such as the user's location and the time the picture was taken.

[0341] Step 2:

[0342] The server processes the received images using an image recognition algorithm to identify the type of waste. The algorithm analyzes the image features and estimates categories such as plastic, glass, and metal.

[0343] Step 3:

[0344] Based on the classification results, the server searches the database for instructions on how to sort the waste and sends them to the user. For example, it might generate an instruction such as, "Put plastics in the blue recycling box."

[0345] Step 4:

[0346] If the emotion engine determines that the user is in an emotionally unstable state, the device generates additional support messages to provide the user with reassuring messages.

[0347] Step 5:

[0348] If the waste requires special handling, the server identifies the nearest appropriate resource recovery company based on the user's location and sends that information to the user.

[0349] Step 6:

[0350] When a user submits a question about recycling or waste disposal in natural language text, the device sends the question to a server. The server analyzes the question, researches relevant information, and creates a thoughtful answer tailored to the user's emotional state, which is then returned to the user.

[0351] Step 7:

[0352] The server records the user's past recycling activities and calculates their environmental contribution based on statistical data. This contribution information is adjusted according to the user's emotional state and displayed to the user as positive feedback from their device.

[0353] Step 8:

[0354] The terminal displays all responses from the server in a user-friendly interface, making it easier for users to select the appropriate waste disposal method. This entire process supports users in effective recycling activities.

[0355] (Example 2)

[0356] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0357] In recent years, as environmental problems have become more serious, proper waste disposal and resource recycling have become crucial. However, a challenge remains in providing sufficient support to help users accurately classify their waste and to motivate them to do so. In particular, there is a need for systems that provide interactive feedback based on user emotions and present specific information on waste collection companies.

[0358] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0359] In this invention, the server includes means for acquiring images of objects via an image acquisition device, means for determining the type of object based on the acquired images using pattern recognition technology, and means for analyzing the user's emotions and generating feedback corresponding to their emotional state. This enables the user to correctly classify waste, understand their environmental contribution, and receive appropriate feedback tailored to their emotions.

[0360] An "image acquisition device" is a device used to acquire images of objects, and primarily refers to a camera.

[0361] "Pattern recognition technology" is a method of analyzing the characteristics of an object from acquired image data and classifying it into known categories.

[0362] "Identifying the type of object" means using pattern recognition technology to recognize the category of a specific object from an image.

[0363] "Emotional analysis" is the process of evaluating a user's emotional state based on their voice and facial expression data.

[0364] "Feedback tailored to emotional state" refers to individually adjusted information and messages provided based on the results of the user's emotional analysis.

[0365] "Providing information on collection companies" means providing users with information about appropriate collection companies based on the type of object identified.

[0366] "Environmental contribution" is a numerical indicator that shows the extent to which the reuse and sorting activities carried out by users contribute to environmental protection.

[0367] This invention is implemented using a dedicated application installed on a user's device. The user starts the system by taking an image of the waste using the device's camera. Simultaneously, the device uses its GPS function to acquire the user's location information and transmits it to the server as metadata along with the image.

[0368] The server uses advanced pattern recognition technology and image recognition algorithms to determine the type of waste from received images. These algorithms include common deep learning models such as ResNet. Based on the classification results, the server provides the user with information on the most suitable sorting method and collection companies via the terminal.

[0369] Furthermore, the device uses a camera and microphone to collect the user's facial expressions and voice, and an emotion engine analyzes the user's emotions. This process utilizes software such as OpenFace and Vokaturi to generate and provide feedback tailored to the user's emotional state.

[0370] For example, if a user takes a picture of a plastic bottle, the server will provide information such as, "This plastic bottle can be recycled. The nearest recycling center is XX." Furthermore, if a user feels that recycling has become a hassle, the emotion engine will evaluate this state and the server will generate an encouraging message such as, "The cumulative effect of recycling activities makes a big difference, let's keep going!"

[0371] Examples of prompts used as input to a generative AI model include: "Please describe the procedure for classifying waste images taken by the user and providing specific sorting methods," and "Please tell me how to generate feedback that responds to the user's emotional state."

[0372] In this way, users can efficiently support environmental protection activities in an interactive and personalized manner.

[0373] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0374] Step 1:

[0375] The user takes an image of the waste using the device. The input is the image taken by the user. The device acquires a high-resolution image using its built-in camera and simultaneously acquires the user's location information using its GPS function. This data is stored as metadata in preparation for subsequent processing. The output is the image data of the waste and the location information.

[0376] Step 2:

[0377] The terminal transmits image data and location information of the waste to the server. Inputs include captured image data and attached location information. The terminal transmits this data to the server via the network and checks its integrity. Outputs are the transmitted image data and location information.

[0378] Step 3:

[0379] The server analyzes the received image data and uses an object recognition algorithm to determine the type of waste. The input is image data sent from the terminal. The server utilizes deep learning models such as ResNet to extract object features from the image and classify them into known categories. The output is information about the type of waste.

[0380] Step 4:

[0381] The server generates appropriate sorting methods and collection company information based on the type of waste and sends it to the terminal. The input is the identified type of waste. The server compares this with past data to determine the optimal sorting guidance and searches for information on relevant collection companies. The output is sorting methods and collection company information customized for the user.

[0382] Step 5:

[0383] The device uses its camera and microphone to send the user's facial expressions and voice to the emotion engine in real time for emotion analysis. The input is the user's facial expressions and voice data as reactions. The device sends this data to the emotion engine, which uses a complex algorithm to estimate the user's emotional state. The output is the user's emotional state data.

[0384] Step 6:

[0385] The server generates and sends feedback messages to the terminal based on the user's emotional data. The input is emotional state data analyzed by the emotion engine. The server customizes encouraging and guiding messages according to the emotional state, creating feedback to boost the user's motivation. The output is the feedback message directed at the user.

[0386] (Application Example 2)

[0387] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0388] In recent years, urban waste management has become a significant social issue, and there is a growing need to promote recycling. However, challenges remain, such as the difficulty for ordinary citizens to understand how to properly sort waste and the difficulty in maintaining motivation. Traditional methods have lacked support tailored to the individual feelings and motivations of users, leading to a decline in willingness to participate in recycling.

[0389] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0390] In this invention, the server includes means for acquiring images of waste via image acquisition means, means for determining the type of waste using image recognition processing based on the acquired images, and means for analyzing the user's emotional state and providing encouragement or instructions based on those emotions. This makes it possible for users to easily sort their waste and to increase their motivation to participate in recycling through feedback tailored to their individual emotions.

[0391] "Image acquisition means" refers to devices and technologies used to capture images of waste.

[0392] "Image recognition processing" refers to algorithms and methods for analyzing specific patterns in acquired images to determine the type of waste.

[0393] "Emotional state analysis" refers to a technology that estimates a user's emotions at a given time based on their facial expressions, voice, etc., and provides appropriate feedback.

[0394] "Providing encouragement or guidance" refers to positive messages or specific action plans sent to users in response to their analyzed emotions.

[0395] "Numerical information on environmental contribution" refers to information that quantifies and visually displays the contribution made by users through their recycling activities.

[0396] "Recording historical data" refers to data management that saves information about past recycling activities and uses it as a reference for the future.

[0397] "City-wide statistical information" refers to information compiled and analyzed from data on recycling activities within a region, in order to understand the overall situation.

[0398] This invention is implemented by an application installed on a user's information processing device. The process begins when the user takes an image of waste and the acquired image is sent to a server. The server receives the image of the waste using an image acquisition means and determines its type through image recognition processing. A generally available artificial intelligence algorithm can be applied to the image recognition processing used in this process.

[0399] The server, based on the type of waste identified from the image, presents the user with the appropriate sorting method. In this process, the user's current location information can also be used to identify the nearest waste collection point. This allows the user to sort their waste in the correct location.

[0400] Furthermore, based on information obtained through the user's camera and microphone, the server analyzes the user's emotional state and provides encouragement and appropriate instructions tailored to the user's feelings. For example, if the server analyzes that the user is feeling stressed, it will send a message such as, "Small daily actions make a big difference to the environment!" as positive feedback regarding recycling activities.

[0401] Users' recycling activity history is stored in a database, and their environmental contribution is calculated. The server can use this historical data to compile statistical information and provide it in a ranking format to raise recycling awareness throughout the city. This allows users to compare their contributions with others and gain further motivation.

[0402] The program's natural language processing uses common natural language generation techniques to provide accurate responses to text questions from users. For example, if a user asks, "How should I dispose of this material?", the system will generate a response such as, "This material is recyclable. The nearest collection point is XX."

[0403] An example of a prompt for a generative AI model, used when performing sentiment analysis, is: "Generate positive feedback based on sentiment monitoring results. The user is recycling and their current sentiment is 'motivated'." This prompt allows the system to generate a message that matches the user's sentiment.

[0404] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0405] Step 1:

[0406] The user takes a picture of the waste using their smartphone camera. The image data is sent to the server as input. In this step, the device's location information is also sent along with the captured image.

[0407] Step 2:

[0408] The server applies an image recognition algorithm to the received image data as input. Data calculations are performed to identify the type of waste, and the recognized waste category is output.

[0409] Step 3:

[0410] The server determines the appropriate waste sorting method based on the classification results and presents the information to the user's terminal. In this step, the server also uses the user's current location information to process data to identify the nearest waste collection point and outputs the results.

[0411] Step 4:

[0412] To analyze the user's emotions, the device's camera and microphone are used to collect data on facial expressions and voice. The server analyzes the input emotion data to determine the user's emotional state.

[0413] Step 5:

[0414] The server generates feedback messages based on emotional states. It utilizes a generation AI model to output appropriate messages corresponding to the input emotional data.

[0415] Step 6:

[0416] On the terminal, users can check their recycling activity history and receive data visualizing their environmental contribution. The server calculates statistical information based on the collected historical data and provides the output as a ranking to the terminal.

[0417] Step 7:

[0418] When a user enters a text question, the server analyzes it using natural language processing technology. Based on the analyzed question, it generates an appropriate answer and sends it to the user's device. The output answer also includes supplementary information based on the user's emotional state.

[0419] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0420] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0421] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0422] [Third Embodiment]

[0423] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0424] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0425] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0426] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0427] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0428] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0429] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0430] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0431] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0432] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0433] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0434] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0435] To implement this invention, the process begins with the user installing a dedicated application on their device. The user uses this application to take an image of the waste and send it to the server. On the server, an image recognition algorithm operates based on the received image to identify the type of waste. This algorithm can, for example, use a machine learning model to analyze features in the image and classify them into categories such as plastic, metal, or glass.

[0436] Based on the identified waste data, the server provides the user with instructions on how to properly sort the waste. This includes indicating the specific type of waste bin and disposal method required. For example, if PET bottles are identified, the server will instruct the user to "put them in the plastic recycling bin."

[0437] Furthermore, if the waste meets certain conditions, the server will provide the user with information on resource recovery companies specializing in waste collection. This includes using the user's location information to select the nearest company and present connection information. For example, if large electronic equipment is identified as waste, a list of companies that collect it can be displayed.

[0438] Users can also ask questions about waste recycling in natural language through the application. The server analyzes these questions and provides relevant information. For example, in response to the question, "Where should I dispose of cardboard?", it will provide a specific answer such as, "Put it in the paper product collection box."

[0439] The server also analyzes the user's recycling history and displays their environmental contribution numerically. This quantified environmental contribution helps maintain motivation for recycling and encourages sustainable environmental behavior. For example, it can show how much CO2 was reduced from the amount of plastic products recycled during a specific period.

[0440] Thus, the present invention realizes an interactive recycling support system that enables users to properly sort waste and easily carry out recycling activities.

[0441] The following describes the processing flow.

[0442] Step 1:

[0443] The user launches a dedicated application and takes a picture of the waste using their device's camera. The captured image is then converted to the optimal format through the application.

[0444] Step 2:

[0445] The device sends the captured image data to the server. At the same time, metadata such as the user's current location and the date and time the image was taken is also sent.

[0446] Step 3:

[0447] The server passes the received image data to an image recognition algorithm to determine the type of waste. The algorithm uses a pre-trained model to extract features from the image and identify the material and category.

[0448] Step 4:

[0449] Based on the identification result, the server retrieves appropriate sorting instructions from the database and generates a message to inform the user. For example, it might create a message such as, "This waste is plastic and should be placed in the recycling bin."

[0450] Step 5:

[0451] The server further identifies a suitable resource recovery company based on the user's current location information if the waste meets certain criteria (e.g., bulky waste or special waste) and provides that information to the user.

[0452] Step 6:

[0453] When a user enters a question, the device sends it to the server. The server uses natural language processing to analyze the question, retrieve relevant information from its database, and generate an answer.

[0454] Step 7:

[0455] The server records users' past recycling activities and calculates their cumulative environmental contribution. By showing users these results in concrete numbers, it aims to improve their recycling awareness.

[0456] Step 8:

[0457] The terminal receives all responses from the server and displays them in an intuitive and easy-to-understand interface for the user. Based on this, the user can properly dispose of the waste.

[0458] (Example 1)

[0459] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0460] Conventional waste disposal systems have struggled to accurately classify materials by type and to provide users with timely and accurate information on appropriate sorting methods and collection companies. Furthermore, there has been a lack of mechanisms to quantitatively demonstrate the environmental contribution of users' recycling activities and to improve users' recycling awareness.

[0461] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0462] In this invention, the server includes means for acquiring images of materials via image acquisition means, means for classifying the type of material using a learning model based on the acquired images, and means for presenting a method for separating the materials based on the classification results. This makes it possible to automatically and accurately classify the type of material and quickly provide the user with appropriate separation methods and information on recycling companies.

[0463] "Image acquisition means" refers to a device or technology for collecting images of materials.

[0464] A "learning model" refers to an algorithm that learns patterns from data and makes predictions and classifications based on new data.

[0465] "Means of classification" refers to techniques for analyzing acquired image data and dividing it into specific categories.

[0466] "Means of providing sorting methods" refers to a function that informs the user of the appropriate processing method and location for each type of material based on its classification.

[0467] A "recycling company" refers to a company or organization that collects materials and carries out appropriate reuse or disposal.

[0468] "Environmental contribution" is a numerical indicator that quantifies the degree of environmental contribution achieved through users' recycling activities.

[0469] "Natural language questions" refer to questions or inquiries about waste disposal that users ask using everyday language.

[0470] "Information provision based on behavioral patterns" refers to a method of analyzing a user's past conscious or unconscious behavioral patterns and proposing optimized information.

[0471] This invention utilizes a dedicated application installed on a user's device. The device uses its camera to capture images of waste and transmits these images to a server via the application. Upon receiving the images, the server uses a learning model to classify the materials within the images. This learning model, for example, employs a machine learning algorithm to analyze image features and classify materials into categories such as plastic, metal, and glass.

[0472] Once classification is complete, the server presents the user with sorting instructions based on the classification results. This information is displayed on the terminal's application screen, making it easy for the user to know which recycling box to place the materials in. Furthermore, the server also provides information on appropriate recycling companies depending on the classified materials. This feature is particularly useful when the materials to be discarded meet certain conditions, and it is possible to display the nearest company using the user's location information.

[0473] Users can also ask questions using natural language through the application on their device. When a user enters a question about how to handle materials, the server analyzes the question and provides relevant information using a generative AI model. For example, in response to the question, "Where should I dispose of cardboard?", it will provide a specific answer such as, "Put it in the paper product recycling box."

[0474] Furthermore, the server can record and analyze the user's past recycling activities as a history, quantifying their environmental contribution and displaying it to the user. Quantifying environmental contribution is important for encouraging sustainable behavior among users. For example, it can display how much CO2 was reduced based on the amount of plastic recycled during a certain period.

[0475] For example, if a user takes a picture of an old newspaper and sends it to the server from the application, the server will identify it as a paper product and provide the user with instructions to "put it in the paper product collection box." Through this process, users can manage their waste more efficiently and effectively.

[0476] Example of a prompt:

[0477] "Please describe the procedure for classifying waste materials as plastic or metal based on images taken by the user."

[0478] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0479] Step 1:

[0480] The user launches a dedicated application on their device and takes a picture of the waste. The input is image data of the waste captured by the device's camera. After taking the picture, the user crops or adjusts the image as needed on the confirmation screen and presses the send button. The output is the prepared image data.

[0481] Step 2:

[0482] The terminal uploads image data sent by the user to the server. The input is the image data after it has been confirmed by the user. The terminal securely transfers this data to the server via an internet connection. The output is the image data received by the server.

[0483] Step 3:

[0484] The server uses a generative AI model to classify the materials within the received image data. The input is the received image data. The server extracts features using an image processing algorithm and inputs these features into the classification model. The model classifies the materials into categories such as plastic and metal, and the output is the classification result.

[0485] Step 4:

[0486] The server presents the user with an appropriate sorting method based on the classification results. The input is the classification result of the materials obtained by the server. The server retrieves sorting methods corresponding to the classified categories from the database and formats them as data for display on the user's terminal. The output is the sorting instruction information sent to the terminal.

[0487] Step 5:

[0488] The server provides information on appropriate waste collection companies based on specific criteria. The inputs are the classification results and the user's location information. The server searches its database of waste collection companies for the relevant waste and selects the most suitable company. The output is the company information displayed on the user's terminal.

[0489] Step 6:

[0490] The user sends recycling-related questions to the server using natural language via the application. The input is the user's question text. The terminal sends this text directly to the server. The output is the question data that reached the server.

[0491] Step 7:

[0492] The server analyzes user questions and provides relevant information using a generated AI model. The input is the user's question text. The server uses natural language processing techniques to analyze the question and generate an appropriate answer. The output is the specific answer information sent to the user.

[0493] Step 8:

[0494] The server analyzes the user's recycling activity history and presents a numerical representation of their environmental contribution. The input is past activity history data. The server retrieves the user's activity records from the database and calculates a quantitative environmental contribution. The output is the numerical environmental contribution displayed on the terminal.

[0495] (Application Example 1)

[0496] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0497] In modern urban life, proper waste classification and disposal are crucial for reducing environmental impact. However, many users lack knowledge of the complex types of waste and their disposal methods, often resulting in incorrect disposal practices. This leads to reduced recycling efficiency, resource waste, and increased environmental burden. Furthermore, collection companies also face challenges in developing collection plans based on accurate waste information, hindering the optimization of resource recovery.

[0498] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0499] In this invention, the server includes means for acquiring images of waste via a camera, means for determining the type of waste using identification technology, and means for performing analysis to optimize waste management on a regional basis. This enables users to easily and accurately classify waste and dispose of it effectively, and enables collection operators to create efficient collection plans.

[0500] A "photography device" is an electronic device used to acquire images of waste, and is equipped with a camera or sensors.

[0501] "Identification technology" refers to techniques that utilize machine learning and artificial intelligence to analyze features within images and accurately determine the type of waste.

[0502] "Classification method" refers to the procedure that specifies the appropriate disposal method according to the type of waste, in order to properly process it.

[0503] A "waste collection business" is an organization that specializes in the collection and disposal of waste, and supports the reuse of resources for the benefit of citizens.

[0504] "Environmental contribution" is an indicator that shows, in concrete numerical terms, how much a waste recycling project has contributed to environmental protection.

[0505] "Regional waste management" refers to a management method aimed at optimizing and improving the efficiency of waste generation, collection, and processing activities within a specific region.

[0506] "Analysis" is the process of scrutinizing information based on data and drawing specific conclusions or recommendations.

[0507] To implement this invention, the user must first install a dedicated application on their device. The device uses its camera function to photograph the waste and sends the image to a cloud server. On the server, an image recognition model operates using a machine learning platform such as TensorFlow. This identifies the type of waste and determines the classification method. The server returns the classification results and information on nearby waste collection companies to the user's device via an API using Flask or FastAPI.

[0508] Through this application, users can ask questions using natural language via voice or text. The server then interprets these questions using natural language processing services such as Dialogflow and provides relevant information. This information includes details on proper waste disposal methods and local recycling stations.

[0509] Furthermore, the server aggregates waste data generated by multiple users and performs analysis to optimize waste management in specific areas. This analysis includes planning efficient collection routes and making suggestions aimed at improving recycling rates.

[0510] As a concrete example, a user takes a photo of unwanted paper products at home with their smartphone and sends it via the application. The server then identifies "paper / cardboard" as the classification result and returns specific instructions to the user, such as "put it in the paper product recycling box." Information on local recycling stations is also provided.

[0511] Examples of prompts to input into a generative AI model:

[0512] "Could you tell me what category this waste belongs to?"

[0513] "Where should I dispose of this waste?"

[0514] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0515] Step 1:

[0516] The user takes an image of the waste using the device's camera function. The input is the actual visual information of the waste, and the output is an image file. This image file serves as the basic data for subsequent identification processing.

[0517] Step 2:

[0518] The device sends the captured image file to the cloud server. The input is an image file, and the output is an upload to the server via the network. The image is transmitted to the server via a secure protocol.

[0519] Step 3:

[0520] The server receives the image file and uses TensorFlow to perform calculations to identify the type of waste in the image. The input is image data, and the output is the waste classification result. In this process, a pre-trained model is used to analyze the features in the image and determine which category the waste belongs to.

[0521] Step 4:

[0522] The server extracts information on appropriate disposal methods and the nearest waste collection service providers based on the classification results of the identified waste. The input is the classification result, and the output is the disposal method and service provider information. This data is retrieved by referencing information stored in a database.

[0523] Step 5:

[0524] Users can input questions about waste in natural language via a terminal. The input is natural language text, which the terminal sends to the server over the network. The output is the user's query data sent to the server.

[0525] Step 6:

[0526] The server uses Dialogflow to interpret natural language questions from the user. The input is natural language text data, and the output is the interpretation result. The server analyzes the user's intent and collects appropriate information from the database.

[0527] Step 7:

[0528] The server provides specific information to the user based on the analyzed results. The input is information extracted based on interpretation, and the output is the response information for the user. The information is sent to the device and displayed on the user's app screen.

[0529] Step 8:

[0530] The server aggregates waste data collected from all users and performs analysis to improve the efficiency of waste management at the regional level. The input is the aggregated waste data, and the output is the data analysis results. These results enable suggestions for optimizing collection routes and improving recycling rates. It also generates example prompts for inputting data into the next generation AI model.

[0531] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0532] This invention is implemented through a dedicated application installed on a user's device. The user uses this application to take images of waste. The device sends the acquired images to a server, which includes the user's current location information and related metadata. The server uses an image recognition algorithm to classify the waste and suggests sorting methods based on the results.

[0533] Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions. By analyzing the user's voice and facial expressions via the device's camera and microphone, it estimates emotions from input data and the user's actions during operation. This engine can determine whether the user is feeling stressed or highly motivated.

[0534] Based on emotional data obtained from the emotion engine, the server provides information and feedback tailored to the user. For example, if a user is dissatisfied with recycling activities, the server will generate more detailed instructions or encouraging messages and display them to the user.

[0535] Furthermore, when a user enters a text question, the device sends the information to the server, which uses natural language processing to interpret the question and generate a response that reflects the user's emotional state. For example, if a user asks, "What should I do with this material?", the server might respond, "That material is recyclable. The nearest recycling box is at XX," and may even add encouraging words.

[0536] This system also provides a means to record the user's recycling history and visualize their cumulative environmental contribution. Depending on the user's sentiment, this environmental contribution information can be provided as positive feedback, supporting the user's sustainable behavior.

[0537] This invention provides an interactive system that allows users to more effectively carry out recycling activities, understand their own contributions, and maintain their motivation.

[0538] The following describes the processing flow.

[0539] Step 1:

[0540] The user launches a dedicated application on their device and takes a picture of the waste with the camera. After taking the picture, the device sends the image data to the server. This includes metadata such as the user's location and the time the picture was taken.

[0541] Step 2:

[0542] The server processes the received images using an image recognition algorithm to identify the type of waste. The algorithm analyzes the image features and estimates categories such as plastic, glass, and metal.

[0543] Step 3:

[0544] Based on the classification results, the server searches the database for instructions on how to sort the waste and sends them to the user. For example, it might generate an instruction such as, "Put plastics in the blue recycling box."

[0545] Step 4:

[0546] If the emotion engine determines that the user is in an emotionally unstable state, the device generates additional support messages to provide the user with reassuring messages.

[0547] Step 5:

[0548] If the waste requires special handling, the server identifies the nearest appropriate resource recovery company based on the user's location and sends that information to the user.

[0549] Step 6:

[0550] When a user submits a question about recycling or waste disposal in natural language text, the device sends the question to a server. The server analyzes the question, researches relevant information, and creates a thoughtful answer tailored to the user's emotional state, which is then returned to the user.

[0551] Step 7:

[0552] The server records the user's past recycling activities and calculates their environmental contribution based on statistical data. This contribution information is adjusted according to the user's emotional state and displayed to the user as positive feedback from their device.

[0553] Step 8:

[0554] The terminal displays all responses from the server in a user-friendly interface, making it easier for users to select the appropriate waste disposal method. This entire process supports users in effective recycling activities.

[0555] (Example 2)

[0556] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0557] In recent years, as environmental problems have become more serious, proper waste disposal and resource recycling have become crucial. However, a challenge remains in providing sufficient support to help users accurately classify their waste and to motivate them to do so. In particular, there is a need for systems that provide interactive feedback based on user emotions and present specific information on waste collection companies.

[0558] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0559] In this invention, the server includes means for acquiring images of objects via an image acquisition device, means for determining the type of object based on the acquired images using pattern recognition technology, and means for analyzing the user's emotions and generating feedback corresponding to their emotional state. This enables the user to correctly classify waste, understand their environmental contribution, and receive appropriate feedback tailored to their emotions.

[0560] An "image acquisition device" is a device used to acquire images of objects, and primarily refers to a camera.

[0561] "Pattern recognition technology" is a method of analyzing the characteristics of an object from acquired image data and classifying it into known categories.

[0562] "Identifying the type of object" means using pattern recognition technology to recognize the category of a specific object from an image.

[0563] "Emotional analysis" is the process of evaluating a user's emotional state based on their voice and facial expression data.

[0564] "Feedback tailored to emotional state" refers to individually adjusted information and messages provided based on the results of the user's emotional analysis.

[0565] "Providing information on collection companies" means providing users with information about appropriate collection companies based on the type of object identified.

[0566] "Environmental contribution" is a numerical indicator that shows the extent to which the reuse and sorting activities carried out by users contribute to environmental protection.

[0567] This invention is implemented using a dedicated application installed on a user's device. The user starts the system by taking an image of the waste using the device's camera. Simultaneously, the device uses its GPS function to acquire the user's location information and transmits it to the server as metadata along with the image.

[0568] The server uses advanced pattern recognition technology and image recognition algorithms to determine the type of waste from received images. These algorithms include common deep learning models such as ResNet. Based on the classification results, the server provides the user with information on the most suitable sorting method and collection companies via the terminal.

[0569] Furthermore, the device uses a camera and microphone to collect the user's facial expressions and voice, and an emotion engine analyzes the user's emotions. This process utilizes software such as OpenFace and Vokaturi to generate and provide feedback tailored to the user's emotional state.

[0570] For example, if a user takes a picture of a plastic bottle, the server will provide information such as, "This plastic bottle can be recycled. The nearest recycling center is XX." Furthermore, if a user feels that recycling has become a hassle, the emotion engine will evaluate this state and the server will generate an encouraging message such as, "The cumulative effect of recycling activities makes a big difference, let's keep going!"

[0571] Examples of prompts used as input to a generative AI model include: "Please describe the procedure for classifying waste images taken by the user and providing specific sorting methods," and "Please tell me how to generate feedback that responds to the user's emotional state."

[0572] In this way, users can efficiently support environmental protection activities in an interactive and personalized manner.

[0573] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0574] Step 1:

[0575] The user takes an image of the waste using the device. The input is the image taken by the user. The device acquires a high-resolution image using its built-in camera and simultaneously acquires the user's location information using its GPS function. This data is stored as metadata in preparation for subsequent processing. The output is the image data of the waste and the location information.

[0576] Step 2:

[0577] The terminal transmits image data and location information of the waste to the server. Inputs include captured image data and attached location information. The terminal transmits this data to the server via the network and checks its integrity. Outputs are the transmitted image data and location information.

[0578] Step 3:

[0579] The server analyzes the received image data and uses an object recognition algorithm to determine the type of waste. The input is image data sent from the terminal. The server utilizes deep learning models such as ResNet to extract object features from the image and classify them into known categories. The output is information about the type of waste.

[0580] Step 4:

[0581] The server generates appropriate sorting methods and collection company information based on the type of waste and sends it to the terminal. The input is the identified type of waste. The server compares this with past data to determine the optimal sorting guidance and searches for information on relevant collection companies. The output is sorting methods and collection company information customized for the user.

[0582] Step 5:

[0583] The device uses its camera and microphone to send the user's facial expressions and voice to the emotion engine in real time for emotion analysis. The input is the user's facial expressions and voice data as reactions. The device sends this data to the emotion engine, which uses a complex algorithm to estimate the user's emotional state. The output is the user's emotional state data.

[0584] Step 6:

[0585] The server generates and sends feedback messages to the terminal based on the user's emotional data. The input is emotional state data analyzed by the emotion engine. The server customizes encouraging and guiding messages according to the emotional state, creating feedback to boost the user's motivation. The output is the feedback message directed at the user.

[0586] (Application Example 2)

[0587] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0588] In recent years, urban waste management has become a significant social issue, and there is a growing need to promote recycling. However, challenges remain, such as the difficulty for ordinary citizens to understand how to properly sort waste and the difficulty in maintaining motivation. Traditional methods have lacked support tailored to the individual feelings and motivations of users, leading to a decline in willingness to participate in recycling.

[0589] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0590] In this invention, the server includes means for acquiring images of waste via image acquisition means, means for determining the type of waste using image recognition processing based on the acquired images, and means for analyzing the user's emotional state and providing encouragement or instructions based on those emotions. This makes it possible for users to easily sort their waste and to increase their motivation to participate in recycling through feedback tailored to their individual emotions.

[0591] "Image acquisition means" refers to devices and technologies used to capture images of waste.

[0592] "Image recognition processing" refers to algorithms and methods for analyzing specific patterns in acquired images to determine the type of waste.

[0593] "Emotional state analysis" refers to a technology that estimates a user's emotions at a given time based on their facial expressions, voice, etc., and provides appropriate feedback.

[0594] "Providing encouragement or guidance" refers to positive messages or specific action plans sent to users in response to their analyzed emotions.

[0595] "Numerical information on environmental contribution" refers to information that quantifies and visually displays the contribution made by users through their recycling activities.

[0596] "Recording historical data" refers to data management that saves information about past recycling activities and uses it as a reference for the future.

[0597] "City-wide statistical information" refers to information compiled and analyzed from data on recycling activities within a region, in order to understand the overall situation.

[0598] This invention is implemented by an application installed on a user's information processing device. The process begins when the user takes an image of waste and the acquired image is sent to a server. The server receives the image of the waste using an image acquisition means and determines its type through image recognition processing. A generally available artificial intelligence algorithm can be applied to the image recognition processing used in this process.

[0599] The server, based on the type of waste identified from the image, presents the user with the appropriate sorting method. In this process, the user's current location information can also be used to identify the nearest waste collection point. This allows the user to sort their waste in the correct location.

[0600] Furthermore, based on information obtained through the user's camera and microphone, the server analyzes the user's emotional state and provides encouragement and appropriate instructions tailored to the user's feelings. For example, if the server analyzes that the user is feeling stressed, it will send a message such as, "Small daily actions make a big difference to the environment!" as positive feedback regarding recycling activities.

[0601] Users' recycling activity history is stored in a database, and their environmental contribution is calculated. The server can use this historical data to compile statistical information and provide it in a ranking format to raise recycling awareness throughout the city. This allows users to compare their contributions with others and gain further motivation.

[0602] The program's natural language processing uses common natural language generation techniques to provide accurate responses to text questions from users. For example, if a user asks, "How should I dispose of this material?", the system will generate a response such as, "This material is recyclable. The nearest collection point is XX."

[0603] An example of a prompt for a generative AI model, used when performing sentiment analysis, is: "Generate positive feedback based on sentiment monitoring results. The user is recycling and their current sentiment is 'motivated'." This prompt allows the system to generate a message that matches the user's sentiment.

[0604] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0605] Step 1:

[0606] The user takes a picture of the waste using their smartphone camera. The image data is sent to the server as input. In this step, the device's location information is also sent along with the captured image.

[0607] Step 2:

[0608] The server applies an image recognition algorithm to the received image data as input. Data calculations are performed to identify the type of waste, and the recognized waste category is output.

[0609] Step 3:

[0610] The server determines the appropriate waste sorting method based on the classification results and presents the information to the user's terminal. In this step, the server also uses the user's current location information to process data to identify the nearest waste collection point and outputs the results.

[0611] Step 4:

[0612] To analyze the user's emotions, the device's camera and microphone are used to collect data on facial expressions and voice. The server analyzes the input emotion data to determine the user's emotional state.

[0613] Step 5:

[0614] The server generates feedback messages based on emotional states. It utilizes a generation AI model to output appropriate messages corresponding to the input emotional data.

[0615] Step 6:

[0616] On the terminal, users can check their recycling activity history and receive data visualizing their environmental contribution. The server calculates statistical information based on the collected historical data and provides the output as a ranking to the terminal.

[0617] Step 7:

[0618] When a user enters a text question, the server analyzes it using natural language processing technology. Based on the analyzed question, it generates an appropriate answer and sends it to the user's device. The output answer also includes supplementary information based on the user's emotional state.

[0619] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0620] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0621] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0622] [Fourth Embodiment]

[0623] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0624] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0625] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0626] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0627] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0628] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0629] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0630] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0631] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0632] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0633] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0634] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0635] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0636] To implement this invention, the process begins with the user installing a dedicated application on their device. The user uses this application to take an image of the waste and send it to the server. On the server, an image recognition algorithm operates based on the received image to identify the type of waste. This algorithm can, for example, use a machine learning model to analyze features in the image and classify them into categories such as plastic, metal, or glass.

[0637] Based on the identified waste data, the server provides the user with instructions on how to properly sort the waste. This includes indicating the specific type of waste bin and disposal method required. For example, if PET bottles are identified, the server will instruct the user to "put them in the plastic recycling bin."

[0638] Furthermore, if the waste meets certain conditions, the server will provide the user with information on resource recovery companies specializing in waste collection. This includes using the user's location information to select the nearest company and present connection information. For example, if large electronic equipment is identified as waste, a list of companies that collect it can be displayed.

[0639] Users can also ask questions about waste recycling in natural language through the application. The server analyzes these questions and provides relevant information. For example, in response to the question, "Where should I dispose of cardboard?", it will provide a specific answer such as, "Put it in the paper product collection box."

[0640] The server also analyzes the user's recycling history and displays their environmental contribution numerically. This quantified environmental contribution helps maintain motivation for recycling and encourages sustainable environmental behavior. For example, it can show how much CO2 was reduced from the amount of plastic products recycled during a specific period.

[0641] Thus, the present invention realizes an interactive recycling support system that enables users to properly sort waste and easily carry out recycling activities.

[0642] The following describes the processing flow.

[0643] Step 1:

[0644] The user launches a dedicated application and takes a picture of the waste using their device's camera. The captured image is then converted to the optimal format through the application.

[0645] Step 2:

[0646] The device sends the captured image data to the server. At the same time, metadata such as the user's current location and the date and time the image was taken is also sent.

[0647] Step 3:

[0648] The server passes the received image data to an image recognition algorithm to determine the type of waste. The algorithm uses a pre-trained model to extract features from the image and identify the material and category.

[0649] Step 4:

[0650] Based on the identification result, the server retrieves appropriate sorting instructions from the database and generates a message to inform the user. For example, it might create a message such as, "This waste is plastic and should be placed in the recycling bin."

[0651] Step 5:

[0652] The server further identifies a suitable resource recovery company based on the user's current location information if the waste meets certain criteria (e.g., bulky waste or special waste) and provides that information to the user.

[0653] Step 6:

[0654] When a user enters a question, the device sends it to the server. The server uses natural language processing to analyze the question, retrieve relevant information from its database, and generate an answer.

[0655] Step 7:

[0656] The server records users' past recycling activities and calculates their cumulative environmental contribution. By showing users these results in concrete numbers, it aims to improve their recycling awareness.

[0657] Step 8:

[0658] The terminal receives all responses from the server and displays them in an intuitive and easy-to-understand interface for the user. Based on this, the user can properly dispose of the waste.

[0659] (Example 1)

[0660] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0661] Conventional waste disposal systems have struggled to accurately classify materials by type and to provide users with timely and accurate information on appropriate sorting methods and collection companies. Furthermore, there has been a lack of mechanisms to quantitatively demonstrate the environmental contribution of users' recycling activities and to improve users' recycling awareness.

[0662] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0663] In this invention, the server includes means for acquiring images of materials via image acquisition means, means for classifying the type of material using a learning model based on the acquired images, and means for presenting a method for separating the materials based on the classification results. This makes it possible to automatically and accurately classify the type of material and quickly provide the user with appropriate separation methods and information on recycling companies.

[0664] "Image acquisition means" refers to a device or technology for collecting images of materials.

[0665] A "learning model" refers to an algorithm that learns patterns from data and makes predictions and classifications based on new data.

[0666] "Means of classification" refers to techniques for analyzing acquired image data and dividing it into specific categories.

[0667] "Means of providing sorting methods" refers to a function that informs the user of the appropriate processing method and location for each type of material based on its classification.

[0668] A "recycling company" refers to a company or organization that collects materials and carries out appropriate reuse or disposal.

[0669] "Environmental contribution" is a numerical indicator that quantifies the degree of environmental contribution achieved through users' recycling activities.

[0670] "Natural language questions" refer to questions or inquiries about waste disposal that users ask using everyday language.

[0671] "Information provision based on behavioral patterns" refers to a method of analyzing a user's past conscious or unconscious behavioral patterns and proposing optimized information.

[0672] This invention utilizes a dedicated application installed on a user's device. The device uses its camera to capture images of waste and transmits these images to a server via the application. Upon receiving the images, the server uses a learning model to classify the materials within the images. This learning model, for example, employs a machine learning algorithm to analyze image features and classify materials into categories such as plastic, metal, and glass.

[0673] Once classification is complete, the server presents the user with sorting instructions based on the classification results. This information is displayed on the terminal's application screen, making it easy for the user to know which recycling box to place the materials in. Furthermore, the server also provides information on appropriate recycling companies depending on the classified materials. This feature is particularly useful when the materials to be discarded meet certain conditions, and it is possible to display the nearest company using the user's location information.

[0674] Users can also ask questions using natural language through the application on their device. When a user enters a question about how to handle materials, the server analyzes the question and provides relevant information using a generative AI model. For example, in response to the question, "Where should I dispose of cardboard?", it will provide a specific answer such as, "Put it in the paper product recycling box."

[0675] Furthermore, the server can record and analyze the user's past recycling activities as a history, quantifying their environmental contribution and displaying it to the user. Quantifying environmental contribution is important for encouraging sustainable behavior among users. For example, it can display how much CO2 was reduced based on the amount of plastic recycled during a certain period.

[0676] For example, if a user takes a picture of an old newspaper and sends it to the server from the application, the server will identify it as a paper product and provide the user with instructions to "put it in the paper product collection box." Through this process, users can manage their waste more efficiently and effectively.

[0677] Example of a prompt:

[0678] "Please describe the procedure for classifying waste materials as plastic or metal based on images taken by the user."

[0679] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0680] Step 1:

[0681] The user launches a dedicated application on their device and takes a picture of the waste. The input is image data of the waste captured by the device's camera. After taking the picture, the user crops or adjusts the image as needed on the confirmation screen and presses the send button. The output is the prepared image data.

[0682] Step 2:

[0683] The terminal uploads image data sent by the user to the server. The input is the image data after it has been confirmed by the user. The terminal securely transfers this data to the server via an internet connection. The output is the image data received by the server.

[0684] Step 3:

[0685] The server uses a generative AI model to classify the materials within the received image data. The input is the received image data. The server extracts features using an image processing algorithm and inputs these features into the classification model. The model classifies the materials into categories such as plastic and metal, and the output is the classification result.

[0686] Step 4:

[0687] The server presents the user with an appropriate sorting method based on the classification results. The input is the classification result of the materials obtained by the server. The server retrieves sorting methods corresponding to the classified categories from the database and formats them as data for display on the user's terminal. The output is the sorting instruction information sent to the terminal.

[0688] Step 5:

[0689] The server provides information on appropriate waste collection companies based on specific criteria. The inputs are the classification results and the user's location information. The server searches its database of waste collection companies for the relevant waste and selects the most suitable company. The output is the company information displayed on the user's terminal.

[0690] Step 6:

[0691] The user sends recycling-related questions to the server using natural language via the application. The input is the user's question text. The terminal sends this text directly to the server. The output is the question data that reached the server.

[0692] Step 7:

[0693] The server analyzes user questions and provides relevant information using a generated AI model. The input is the user's question text. The server uses natural language processing techniques to analyze the question and generate an appropriate answer. The output is the specific answer information sent to the user.

[0694] Step 8:

[0695] The server analyzes the user's recycling activity history and presents a numerical representation of their environmental contribution. The input is past activity history data. The server retrieves the user's activity records from the database and calculates a quantitative environmental contribution. The output is the numerical environmental contribution displayed on the terminal.

[0696] (Application Example 1)

[0697] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0698] In modern urban life, proper waste classification and disposal are crucial for reducing environmental impact. However, many users lack knowledge of the complex types of waste and their disposal methods, often resulting in incorrect disposal practices. This leads to reduced recycling efficiency, resource waste, and increased environmental burden. Furthermore, collection companies also face challenges in developing collection plans based on accurate waste information, hindering the optimization of resource recovery.

[0699] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0700] In this invention, the server includes means for acquiring images of waste via a camera, means for determining the type of waste using identification technology, and means for performing analysis to optimize waste management on a regional basis. This enables users to easily and accurately classify waste and dispose of it effectively, and enables collection operators to create efficient collection plans.

[0701] A "photography device" is an electronic device used to acquire images of waste, and is equipped with a camera or sensors.

[0702] "Identification technology" refers to techniques that utilize machine learning and artificial intelligence to analyze features within images and accurately determine the type of waste.

[0703] "Classification method" refers to the procedure that specifies the appropriate disposal method according to the type of waste, in order to properly process it.

[0704] A "waste collection business" is an organization that specializes in the collection and disposal of waste, and supports the reuse of resources for the benefit of citizens.

[0705] "Environmental contribution" is an indicator that shows, in concrete numerical terms, how much a waste recycling project has contributed to environmental protection.

[0706] "Regional waste management" refers to a management method aimed at optimizing and improving the efficiency of waste generation, collection, and processing activities within a specific region.

[0707] "Analysis" is the process of scrutinizing information based on data and drawing specific conclusions or recommendations.

[0708] To implement this invention, the user must first install a dedicated application on their device. The device uses its camera function to photograph the waste and sends the image to a cloud server. On the server, an image recognition model operates using a machine learning platform such as TensorFlow. This identifies the type of waste and determines the classification method. The server returns the classification results and information on nearby waste collection companies to the user's device via an API using Flask or FastAPI.

[0709] Through this application, users can ask questions using natural language via voice or text. The server then interprets these questions using natural language processing services such as Dialogflow and provides relevant information. This information includes details on proper waste disposal methods and local recycling stations.

[0710] Furthermore, the server aggregates waste data generated by multiple users and performs analysis to optimize waste management in specific areas. This analysis includes planning efficient collection routes and making suggestions aimed at improving recycling rates.

[0711] As a concrete example, a user takes a photo of unwanted paper products at home with their smartphone and sends it via the application. The server then identifies "paper / cardboard" as the classification result and returns specific instructions to the user, such as "put it in the paper product recycling box." Information on local recycling stations is also provided.

[0712] Examples of prompts to input into a generative AI model:

[0713] "Could you tell me what category this waste belongs to?"

[0714] "Where should I dispose of this waste?"

[0715] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0716] Step 1:

[0717] The user takes an image of the waste using the device's camera function. The input is the actual visual information of the waste, and the output is an image file. This image file serves as the basic data for subsequent identification processing.

[0718] Step 2:

[0719] The device sends the captured image file to the cloud server. The input is an image file, and the output is an upload to the server via the network. The image is transmitted to the server via a secure protocol.

[0720] Step 3:

[0721] The server receives the image file and uses TensorFlow to perform calculations to identify the type of waste in the image. The input is image data, and the output is the waste classification result. In this process, a pre-trained model is used to analyze the features in the image and determine which category the waste belongs to.

[0722] Step 4:

[0723] The server extracts information on appropriate disposal methods and the nearest waste collection service providers based on the classification results of the identified waste. The input is the classification result, and the output is the disposal method and service provider information. This data is retrieved by referencing information stored in a database.

[0724] Step 5:

[0725] Users can input questions about waste in natural language via a terminal. The input is natural language text, which the terminal sends to the server over the network. The output is the user's query data sent to the server.

[0726] Step 6:

[0727] The server uses Dialogflow to interpret natural language questions from the user. The input is natural language text data, and the output is the interpretation result. The server analyzes the user's intent and collects appropriate information from the database.

[0728] Step 7:

[0729] The server provides specific information to the user based on the analyzed results. The input is information extracted based on interpretation, and the output is the response information for the user. The information is sent to the device and displayed on the user's app screen.

[0730] Step 8:

[0731] The server aggregates waste data collected from all users and performs analysis to improve the efficiency of waste management at the regional level. The input is the aggregated waste data, and the output is the data analysis results. These results enable suggestions for optimizing collection routes and improving recycling rates. It also generates example prompts for inputting data into the next generation AI model.

[0732] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0733] This invention is implemented through a dedicated application installed on a user's device. The user uses this application to take images of waste. The device sends the acquired images to a server, which includes the user's current location information and related metadata. The server uses an image recognition algorithm to classify the waste and suggests sorting methods based on the results.

[0734] Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions. By analyzing the user's voice and facial expressions via the device's camera and microphone, it estimates emotions from input data and the user's actions during operation. This engine can determine whether the user is feeling stressed or highly motivated.

[0735] Based on emotional data obtained from the emotion engine, the server provides information and feedback tailored to the user. For example, if a user is dissatisfied with recycling activities, the server will generate more detailed instructions or encouraging messages and display them to the user.

[0736] Furthermore, when a user enters a text question, the device sends the information to the server, which uses natural language processing to interpret the question and generate a response that reflects the user's emotional state. For example, if a user asks, "What should I do with this material?", the server might respond, "That material is recyclable. The nearest recycling box is at XX," and may even add encouraging words.

[0737] This system also provides a means to record the user's recycling history and visualize their cumulative environmental contribution. Depending on the user's sentiment, this environmental contribution information can be provided as positive feedback, supporting the user's sustainable behavior.

[0738] This invention provides an interactive system that allows users to more effectively carry out recycling activities, understand their own contributions, and maintain their motivation.

[0739] The following describes the processing flow.

[0740] Step 1:

[0741] The user launches a dedicated application on their device and takes a picture of the waste with the camera. After taking the picture, the device sends the image data to the server. This includes metadata such as the user's location and the time the picture was taken.

[0742] Step 2:

[0743] The server processes the received images using an image recognition algorithm to identify the type of waste. The algorithm analyzes the image features and estimates categories such as plastic, glass, and metal.

[0744] Step 3:

[0745] Based on the classification results, the server searches the database for instructions on how to sort the waste and sends them to the user. For example, it might generate an instruction such as, "Put plastics in the blue recycling box."

[0746] Step 4:

[0747] If the emotion engine determines that the user is in an emotionally unstable state, the device generates additional support messages to provide the user with reassuring messages.

[0748] Step 5:

[0749] If the waste requires special handling, the server identifies the nearest appropriate resource recovery company based on the user's location and sends that information to the user.

[0750] Step 6:

[0751] When a user submits a question about recycling or waste disposal in natural language text, the device sends the question to a server. The server analyzes the question, researches relevant information, and creates a thoughtful answer tailored to the user's emotional state, which is then returned to the user.

[0752] Step 7:

[0753] The server records the user's past recycling activities and calculates their environmental contribution based on statistical data. This contribution information is adjusted according to the user's emotional state and displayed to the user as positive feedback from their device.

[0754] Step 8:

[0755] The terminal displays all responses from the server in a user-friendly interface, making it easier for users to select the appropriate waste disposal method. This entire process supports users in effective recycling activities.

[0756] (Example 2)

[0757] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0758] In recent years, as environmental problems have become more serious, proper waste disposal and resource recycling have become crucial. However, a challenge remains in providing sufficient support to help users accurately classify their waste and to motivate them to do so. In particular, there is a need for systems that provide interactive feedback based on user emotions and present specific information on waste collection companies.

[0759] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0760] In this invention, the server includes means for acquiring images of objects via an image acquisition device, means for determining the type of object based on the acquired images using pattern recognition technology, and means for analyzing the user's emotions and generating feedback corresponding to their emotional state. This enables the user to correctly classify waste, understand their environmental contribution, and receive appropriate feedback tailored to their emotions.

[0761] An "image acquisition device" is a device used to acquire images of objects, and primarily refers to a camera.

[0762] "Pattern recognition technology" is a method of analyzing the characteristics of an object from acquired image data and classifying it into known categories.

[0763] "Identifying the type of object" means using pattern recognition technology to recognize the category of a specific object from an image.

[0764] "Emotional analysis" is the process of evaluating a user's emotional state based on their voice and facial expression data.

[0765] "Feedback tailored to emotional state" refers to individually adjusted information and messages provided based on the results of the user's emotional analysis.

[0766] "Providing information on collection companies" means providing users with information about appropriate collection companies based on the type of object identified.

[0767] "Environmental contribution" is a numerical indicator that shows the extent to which the reuse and sorting activities carried out by users contribute to environmental protection.

[0768] This invention is implemented using a dedicated application installed on a user's device. The user starts the system by taking an image of the waste using the device's camera. Simultaneously, the device uses its GPS function to acquire the user's location information and transmits it to the server as metadata along with the image.

[0769] The server uses advanced pattern recognition technology and image recognition algorithms to determine the type of waste from received images. These algorithms include common deep learning models such as ResNet. Based on the classification results, the server provides the user with information on the most suitable sorting method and collection companies via the terminal.

[0770] Furthermore, the device uses a camera and microphone to collect the user's facial expressions and voice, and an emotion engine analyzes the user's emotions. This process utilizes software such as OpenFace and Vokaturi to generate and provide feedback tailored to the user's emotional state.

[0771] For example, if a user takes a picture of a plastic bottle, the server will provide information such as, "This plastic bottle can be recycled. The nearest recycling center is XX." Furthermore, if a user feels that recycling has become a hassle, the emotion engine will evaluate this state and the server will generate an encouraging message such as, "The cumulative effect of recycling activities makes a big difference, let's keep going!"

[0772] Examples of prompts used as input to a generative AI model include: "Please describe the procedure for classifying waste images taken by the user and providing specific sorting methods," and "Please tell me how to generate feedback that responds to the user's emotional state."

[0773] In this way, users can efficiently support environmental protection activities in an interactive and personalized manner.

[0774] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0775] Step 1:

[0776] The user takes an image of the waste using the device. The input is the image taken by the user. The device acquires a high-resolution image using its built-in camera and simultaneously acquires the user's location information using its GPS function. This data is stored as metadata in preparation for subsequent processing. The output is the image data of the waste and the location information.

[0777] Step 2:

[0778] The terminal transmits image data and location information of the waste to the server. Inputs include captured image data and attached location information. The terminal transmits this data to the server via the network and checks its integrity. Outputs are the transmitted image data and location information.

[0779] Step 3:

[0780] The server analyzes the received image data and uses an object recognition algorithm to determine the type of waste. The input is image data sent from the terminal. The server utilizes deep learning models such as ResNet to extract object features from the image and classify them into known categories. The output is information about the type of waste.

[0781] Step 4:

[0782] The server generates appropriate sorting methods and collection company information based on the type of waste and sends it to the terminal. The input is the identified type of waste. The server compares this with past data to determine the optimal sorting guidance and searches for information on relevant collection companies. The output is sorting methods and collection company information customized for the user.

[0783] Step 5:

[0784] The device uses its camera and microphone to send the user's facial expressions and voice to the emotion engine in real time for emotion analysis. The input is the user's facial expressions and voice data as reactions. The device sends this data to the emotion engine, which uses a complex algorithm to estimate the user's emotional state. The output is the user's emotional state data.

[0785] Step 6:

[0786] The server generates and sends feedback messages to the terminal based on the user's emotional data. The input is emotional state data analyzed by the emotion engine. The server customizes encouraging and guiding messages according to the emotional state, creating feedback to boost the user's motivation. The output is the feedback message directed at the user.

[0787] (Application Example 2)

[0788] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0789] In recent years, urban waste management has become a significant social issue, and there is a growing need to promote recycling. However, challenges remain, such as the difficulty for ordinary citizens to understand how to properly sort waste and the difficulty in maintaining motivation. Traditional methods have lacked support tailored to the individual feelings and motivations of users, leading to a decline in willingness to participate in recycling.

[0790] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0791] In this invention, the server includes means for acquiring images of waste via image acquisition means, means for determining the type of waste using image recognition processing based on the acquired images, and means for analyzing the user's emotional state and providing encouragement or instructions based on those emotions. This makes it possible for users to easily sort their waste and to increase their motivation to participate in recycling through feedback tailored to their individual emotions.

[0792] "Image acquisition means" refers to devices and technologies used to capture images of waste.

[0793] "Image recognition processing" refers to algorithms and methods for analyzing specific patterns in acquired images to determine the type of waste.

[0794] "Emotional state analysis" refers to a technology that estimates a user's emotions at a given time based on their facial expressions, voice, etc., and provides appropriate feedback.

[0795] "Providing encouragement or guidance" refers to positive messages or specific action plans sent to users in response to their analyzed emotions.

[0796] "Numerical information on environmental contribution" refers to information that quantifies and visually displays the contribution made by users through their recycling activities.

[0797] "Recording historical data" refers to data management that saves information about past recycling activities and uses it as a reference for the future.

[0798] "City-wide statistical information" refers to information compiled and analyzed from data on recycling activities within a region, in order to understand the overall situation.

[0799] This invention is implemented by an application installed on a user's information processing device. The process begins when the user takes an image of waste and the acquired image is sent to a server. The server receives the image of the waste using an image acquisition means and determines its type through image recognition processing. A generally available artificial intelligence algorithm can be applied to the image recognition processing used in this process.

[0800] The server, based on the type of waste identified from the image, presents the user with the appropriate sorting method. In this process, the user's current location information can also be used to identify the nearest waste collection point. This allows the user to sort their waste in the correct location.

[0801] Furthermore, based on information obtained through the user's camera and microphone, the server analyzes the user's emotional state and provides encouragement and appropriate instructions tailored to the user's feelings. For example, if the server analyzes that the user is feeling stressed, it will send a message such as, "Small daily actions make a big difference to the environment!" as positive feedback regarding recycling activities.

[0802] Users' recycling activity history is stored in a database, and their environmental contribution is calculated. The server can use this historical data to compile statistical information and provide it in a ranking format to raise recycling awareness throughout the city. This allows users to compare their contributions with others and gain further motivation.

[0803] The program's natural language processing uses common natural language generation techniques to provide accurate responses to text questions from users. For example, if a user asks, "How should I dispose of this material?", the system will generate a response such as, "This material is recyclable. The nearest collection point is XX."

[0804] An example of a prompt for a generative AI model, used when performing sentiment analysis, is: "Generate positive feedback based on sentiment monitoring results. The user is recycling and their current sentiment is 'motivated'." This prompt allows the system to generate a message that matches the user's sentiment.

[0805] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0806] Step 1:

[0807] The user takes a picture of the waste using their smartphone camera. The image data is sent to the server as input. In this step, the device's location information is also sent along with the captured image.

[0808] Step 2:

[0809] The server applies an image recognition algorithm to the received image data as input. Data calculations are performed to identify the type of waste, and the recognized waste category is output.

[0810] Step 3:

[0811] The server determines the appropriate waste sorting method based on the classification results and presents the information to the user's terminal. In this step, the server also uses the user's current location information to process data to identify the nearest waste collection point and outputs the results.

[0812] Step 4:

[0813] To analyze the user's emotions, the device's camera and microphone are used to collect data on facial expressions and voice. The server analyzes the input emotion data to determine the user's emotional state.

[0814] Step 5:

[0815] The server generates feedback messages based on emotional states. It utilizes a generation AI model to output appropriate messages corresponding to the input emotional data.

[0816] Step 6:

[0817] On the terminal, users can check their recycling activity history and receive data visualizing their environmental contribution. The server calculates statistical information based on the collected historical data and provides the output as a ranking to the terminal.

[0818] Step 7:

[0819] When a user enters a text question, the server analyzes it using natural language processing technology. Based on the analyzed question, it generates an appropriate answer and sends it to the user's device. The output answer also includes supplementary information based on the user's emotional state.

[0820] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0821] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0822] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0823] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0824] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0825] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0826] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0827] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0828] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0829] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0830] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0831] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0832] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0833] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0834] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0835] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0836] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0837] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0838] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0839] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0840] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0841] The following is further disclosed regarding the embodiments described above.

[0842] (Claim 1)

[0843] A means for acquiring images of waste via an image acquisition device,

[0844] A method for identifying the type of waste using an image recognition algorithm based on acquired images,

[0845] A means of presenting waste sorting methods based on the classification results,

[0846] A means of identifying and providing information on resource recovery companies that are suitable for the type of waste,

[0847] A means of displaying to users the environmental contribution of waste recycling as numerical information,

[0848] A system that includes this.

[0849] (Claim 2)

[0850] The system according to claim 1, comprising means for analyzing text-based questions from users and providing appropriate answers in natural language.

[0851] (Claim 3)

[0852] The system according to claim 1, comprising means for collecting historical data on waste sorting and recycling activities and optimizing information provision based on user behavior patterns.

[0853] "Example 1"

[0854] (Claim 1)

[0855] A means for acquiring an image of a material via an image acquisition means,

[0856] A method for classifying material types using a learning model based on acquired images,

[0857] A means of presenting a method for separating materials based on the classification results,

[0858] A means of identifying and providing information on recycling companies suitable for the type of material,

[0859] A means of displaying to users the environmental contribution of material recycling as numerical information,

[0860] A means of analyzing natural language questions from users and providing relevant information,

[0861] A system that includes this.

[0862] (Claim 2)

[0863] The system according to claim 1, comprising means for analyzing natural language questions from users and providing appropriate answers in natural language.

[0864] (Claim 3)

[0865] The system according to claim 1, comprising means for collecting historical data on material sorting and recycling activities and optimizing information provision based on user behavior patterns.

[0866] "Application Example 1"

[0867] (Claim 1)

[0868] A means for acquiring images of waste via a camera,

[0869] A means of determining the type of waste by using identification technology based on acquired images,

[0870] A means of presenting a method for classifying waste based on the judgment results,

[0871] A means of identifying and providing information on collection businesses that are suitable for the type of waste,

[0872] A means of displaying to users the environmental contribution of waste recycling as numerical information,

[0873] A means of conducting analysis to optimize waste management at the regional level to which each household or facility belongs,

[0874] A system that includes this.

[0875] (Claim 2)

[0876] The system according to claim 1, comprising means for interpreting natural language inquiries from users and providing appropriate information.

[0877] (Claim 3)

[0878] The system according to claim 1, which collects record data on waste classification and recycling activities and controls means for optimizing information provision based on user behavior patterns.

[0879] "Example 2 of combining an emotion engine"

[0880] (Claim 1)

[0881] Means for acquiring an image of an object via an image acquisition device,

[0882] A method for determining the type of object by utilizing pattern recognition technology based on acquired images,

[0883] A means for presenting a method for classifying objects based on the discrimination result,

[0884] A means of identifying a collection company suitable for the type of object and presenting relevant information,

[0885] A means of displaying to users the environmental contribution of object recycling as numerical information,

[0886] A means of analyzing the user's emotions and generating feedback according to their emotional state,

[0887] A system that includes this.

[0888] (Claim 2)

[0889] The system according to claim 1, comprising means for analyzing a textual question from a user and providing an appropriate answer in natural language.

[0890] (Claim 3)

[0891] The system according to claim 1, comprising means for collecting historical data on object classification and reuse activities, and optimizing information provision based on user behavior patterns.

[0892] "Application example 2 when combining with an emotional engine"

[0893] (Claim 1)

[0894] A means for acquiring images of waste via an image acquisition means,

[0895] A method for identifying the type of waste using image recognition processing based on acquired images,

[0896] A means of presenting waste sorting methods based on the classification results,

[0897] A means of identifying resource recovery businesses that are suitable for the type of waste and providing information on them,

[0898] A means of analyzing the user's emotional state and providing emotion-based encouragement or guidance,

[0899] A means of displaying to users the environmental contribution of waste recycling as numerical information,

[0900] A means of recording users' recycling activity history data and displaying rankings using city-wide statistical information,

[0901] A system that includes this.

[0902] (Claim 2)

[0903] The system according to claim 1, comprising means for analyzing text-based questions from users and providing appropriate answers in natural language.

[0904] (Claim 3)

[0905] The system according to claim 1, comprising means for collecting historical data on waste sorting and recycling activities, optimizing information provision based on user behavior patterns, and generating emotion-responsive feedback. [Explanation of Symbols]

[0906] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for acquiring images of waste via a camera, A means of determining the type of waste by using identification technology based on acquired images, A means of presenting a method for classifying waste based on the judgment results, A means of identifying and providing information on collection businesses that are suitable for the type of waste, A means of displaying to users the environmental contribution of waste recycling as numerical information, A means of conducting analysis to optimize waste management at the regional level to which each household or facility belongs, A system that includes this.

2. The system according to claim 1, comprising means for interpreting natural language inquiries from users and providing appropriate information.

3. The system according to claim 1, which collects record data on waste classification and recycling activities and controls means for optimizing information provision based on user behavior patterns.

Images