system

Abstract

We provide the system. [Solution] A means of retrieving the schedule of community activities from a storage device and sending notifications to residents, A means of tracking the location information of waste collection equipment and providing it to residents immediately, A means of automatically adjusting and notifying participants of meeting schedules for local groups based on their availability, A means of converting audio information from a meeting into text information and generating meeting minutes, A means of receiving health information from the elderly and providing advice by comparing it with standard values, A means of providing health monitoring functions to residents' devices and notifying them of medical-related information, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Activities in local communities tend to become inefficient due to the aging of residents and a shortage of manpower. Also, there are a wide variety of daily tasks such as schedule management of garbage collection, operation of neighborhood associations, and health support for the elderly, and the burden of these tasks on residents and administrative staff has been an issue. In order to address regional depopulation and improve the quality of life of residents, it is required to streamline and automate these tasks.

Means for Solving the Problems

[0005] The present invention provides a means to automate and streamline various aspects of community activities. Specifically, it includes means for retrieving community activity schedules from a database and automatically sending notifications to residents, and means for sharing garbage truck tracking information with residents. It also provides means for adjusting neighborhood association meeting schedules based on participants' schedules and automatically generating meeting minutes using audio data. Furthermore, it includes means for analyzing health data entered by elderly people and providing advice by comparing it with standard values. In this way, the system can reduce the burden on residents and administrative staff and improve the operational efficiency of the community.

[0006] A "database" is a system that stores information in a structured form and allows for efficient searching and updating as needed.

[0007] "Community activities" refer to social, cultural, and environmental activities carried out jointly by residents and related organizations within a specific geographical area.

[0008] "Residents" refer to individuals who live in a specific area and are the target or participants in community activities.

[0009] "Notification" refers to a message or alert system sent to inform a recipient of specific information.

[0010] A "garbage truck" refers to a vehicle that collects waste generated within a community and transports it to an appropriate processing facility.

[0011] "Tracking information" refers to data used to understand the location and movement of a specific object or individual.

[0012] A "neighborhood association" refers to a community of local residents that operates with the purpose of resolving local issues and promoting local activities.

[0013] A "meeting schedule" refers to the details of a specific meeting, such as the date, time, and location, and is used for managing the schedules of those involved.

[0014] "Voice data" refers to digital information obtained by converting sound vibrations into electrical signals, and is used for voice analysis and recording.

[0015] "Proceedings" refers to a document that records the content of a meeting or gathering, and is used to confirm the resolutions and key points of the discussion at a later date.

[0016] "Health data" refers to information indicating the state of an individual's body, including numerical values such as body temperature, blood pressure, and pulse.

[0017] "Standard value" refers to a value within a generally recognized normal range for a specific parameter, and serves as a criterion for evaluating health status.

[0018] "Advice" refers to the act of providing professional or practical solutions or guidance for problems or questions.

Brief Description of the Drawings

[0019] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] Shows an emotion map where multiple emotions are mapped. [Figure 10] Shows an emotion map where multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Modes for Carrying Out the Invention

[0020] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0023] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0024] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0025] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0027] [First Embodiment]

[0028] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0029] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0032] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0035] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0039] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0040] This invention provides an AI agent system that supports community activities. This system primarily functions in three areas: garbage management, neighborhood association operation support, and elderly care support.

[0041] First, the server extracts garbage collection schedule information from a local activity database and sends a notification to residents' devices based on this information. Residents' devices receive this notification and display a reminder at the set time. This entire process helps residents put out their garbage on the correct dates and times. For example, if combustible waste is collected on Mondays and Thursdays, the server automatically notifies residents, and their devices send a reminder in the morning.

[0042] Next, the server receives location data from GPS devices installed on garbage trucks and makes it accessible to residents as map information. Residents can then view this information through an application and monitor in real time whether their garbage is being collected properly. For example, when a resident opens the app, they can see that a garbage truck is heading towards their area.

[0043] The neighborhood association's operational support system incorporates a server that checks each participant's availability, automatically adjusts meeting schedules, and sends notifications. Furthermore, during meetings, audio data is automatically transcribed, generating meeting minutes. These minutes are shared with participants, who can review them on their devices and make corrections as needed.

[0044] Finally, as part of support for the elderly, a system has been introduced that allows users to input daily health data using a terminal and send it to a server. The server analyzes this data, compares it to standard values, evaluates the user's health status, and generates advice. For example, if body temperature or blood pressure exceeds a certain range, a health alert is automatically sent.

[0045] This system is designed to streamline local community activities and reduce the burden on residents and stakeholders. In this way, it can help address depopulation in the region and strengthen cooperation among residents.

[0046] The following describes the processing flow.

[0047] Step 1: The server accesses the local activity schedule database to retrieve the latest garbage collection schedule information.

[0048] Step 2: The server uses the residents' contact information to send the retrieved garbage collection schedule as a push notification to each resident's device.

[0049] Step 3: Set up the device to display received notifications and provide reminders to residents at the specified time.

[0050] Step 4: The user checks the notification on their device and prepares to take out the trash if necessary.

[0051] Step 5: The server collects location information in real time from GPS devices installed in the garbage trucks.

[0052] Step 6: The server analyzes the location information it has collected and formats the data to display the route of the collection vehicle on a map.

[0053] Step 7: Users check the current location and planned route of the garbage truck via a smartphone app.

[0054] Step 8: For neighborhood association operation support, the server retrieves participants' calendar information and automatically adjusts the optimal meeting schedule.

[0055] Step 9: The server notifies participants of the meeting date as a calendar entry on their devices.

[0056] Step 10: The user checks the meeting schedule on their device and makes manual changes if necessary.

[0057] Step 11: During the meeting, the user collects audio data using a recording device.

[0058] Step 12: The server receives the recorded audio data and converts it into text data using speech recognition technology.

[0059] Step 13: The server checks the converted text data and automatically generates the first version of the meeting minutes.

[0060] Step 14: The user reviews the initial version of the meeting minutes, makes any necessary revisions, and finalizes the official version.

[0061] Step 15: Every morning, the elderly person uses a device to input health data (e.g., body temperature, blood pressure) and sends it to the server.

[0062] Step 16: The server analyzes the received health data and evaluates the health status by comparing it to standard health values.

[0063] Step 17: Based on the evaluation results, the server generates necessary advice and notifies the user's terminal.

[0064] Step 18: Review the advice the user received and use it to help manage their health.

[0065] (Example 1)

[0066] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0067] In local communities, there is a need for systems that efficiently support diverse activities in which residents participate and the health management of the elderly. However, existing methods suffer from delays in information dissemination, difficulties in coordination, and limitations in providing individualized support. Therefore, efficiently managing information related to community activities and providing appropriate support to each resident is a challenge.

[0068] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0069] In this invention, the server includes means for acquiring scheduled information on local activities from an information storage means via a communication medium and transmitting notifications to a receiving device; means for collecting location information of mobile objects and providing it to residents immediately; and means for automatically adjusting and notifying participants of the meeting date and time of local organizations based on their available time. This enables the rapid provision of information on local activities, the smooth coordination of various events, and the efficient provision of individual support.

[0070] "Communication medium" refers to the means and routes for sending and receiving information, and is a general term that includes wired and wireless technologies.

[0071] "Information storage means" refers to devices and systems used to store and manage data, such as databases.

[0072] A "receiving device" refers to a device used to receive information transmitted from a server, and includes smartphones and tablets.

[0073] "Location information of a moving object" refers to data indicating the geographical location of a specific object or vehicle, and is obtained using technologies such as GPS.

[0074] A "community group" refers to a collection or organization of residents in a specific geographical area, such as a neighborhood association or a local residents' association.

[0075] "Meeting date and time" is a term that refers to the date and time when a particular activity or event takes place.

[0076] "Available time" refers to information indicating the dates and times when an individual or group can participate in a particular activity.

[0077] "Means of notification" refers to the methods and technologies used to transmit information to recipients, such as email and push notifications.

[0078] This invention is a system for efficiently supporting community activities, and mainly consists of a server, terminals, and users. The server accesses a region-specific information storage means and obtains scheduled information on community activities. This information is mainly managed using a database management system. The server transmits the obtained information as a notification to a receiving device via a communication medium. The notification is provided via an internet-based protocol (e.g., HTTP) and communication service. As a specific example, it is conceivable that a reminder of neighborhood association events would be sent to residents' terminals every Monday.

[0079] Furthermore, the server collects location information of moving objects and provides this information to residents in real time. This involves data processing using GPS devices and map APIs. The terminal can then visually present this information to residents through an application. For example, when a resident opens the app, the current location of the garbage truck is displayed on a map.

[0080] Furthermore, to support the efficient management of local group meetings, the server collects each participant's availability and automatically adjusts the optimal meeting time. This utilizes a calendar API to aggregate individual schedule information and perform calculations to derive the optimal solution. During the meeting, audio information is acquired using recording devices and converted into text using speech recognition technology. The generated text information, which serves as meeting minutes, is shared with users' devices for later review and correction.

[0081] Finally, health information is entered by the elderly person via their device and sent to the server. Within the server, a data analysis program compares the health information to standard values ​​and assesses their health status. The assessment results and any necessary advice are then notified to the user's device. This could potentially result in an alert being sent to the elderly person, such as, "Your body temperature is high today, please rest."

[0082] An example of a prompt message is, "Please briefly explain the process for automatically scheduling neighborhood association meetings." This invention provides an effective means to comprehensively support the lives of residents and facilitate community activities.

[0083] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0084] Step 1:

[0085] The server accesses a database of local activities and extracts schedule information relevant to residents. It uses tables and query conditions within the database as input and retrieves the schedule information in JSON format as output. Based on this information, the server prepares notifications and organizes the data for sending them via communication media. Specifically, it extracts information such as "The next neighborhood association meeting is next Wednesday" from the database.

[0086] Step 2:

[0087] The device receives notifications sent from the server and displays reminders at the appropriate time based on the user's schedule. It receives notification data from the server as input and displays reminders on the user's screen as output. Specifically, a pop-up message appears on the device saying, "There is a neighborhood association meeting tomorrow."

[0088] Step 3:

[0089] The server receives current location information from the garbage truck's GPS device. It takes location data from the GPS as input and updates that data as map information in real time as output. The server uses a map API to generate visual location information and provides it to the terminal. Specifically, it displays "Garbage truck passing through area A" on the map.

[0090] Step 4:

[0091] The device displays map information provided by the server, allowing the user to check the location of the garbage truck in real time. It receives map data from the server as input and displays routes and location icons on the map application as output. In practice, the app displays updated information such as "10 minutes until the garbage truck arrives."

[0092] Step 5:

[0093] The server references the calendars of meeting participants and calculates the optimal meeting date. It takes participants' calendar information (a list of their free time slots) as input and generates a new, adjusted meeting date as output. Specifically, it calculates information such as, "The next meeting is scheduled for Friday at 3 PM."

[0094] Step 6:

[0095] The device receives updated meeting schedules from the server and notifies participants. It takes the new meeting schedule as input and adds it as a calendar event as output. Specifically, the device's calendar app receives a notification stating, "A new meeting has been added."

[0096] Step 7:

[0097] The user uses their device to input their health information and sends it to the server. Inputs include health indicators such as body temperature and blood pressure, and output is health data sent to the server. For example, the user might enter "36.7 degrees" into the app and tap the send button.

[0098] Step 8:

[0099] The server analyzes the received health data and evaluates the user's health status by comparing it to standard values. It uses health data submitted by the user as input and generates analysis results and advice as output. For example, it might create a message such as, "Your blood pressure is a little high, so please drink more water."

[0100] Step 9:

[0101] The device receives health analysis results from the server and displays an alert to the user. It receives advice messages from the server as input and displays a warning directly on the user's screen as output. Specifically, the app's notification area displays "Health Warning: Your body temperature is high today, please be careful."

[0102] (Application Example 1)

[0103] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0104] To efficiently manage and promote community activities, the rapid provision of various information and smooth communication are essential. However, conventional methods have problems such as missing garbage collection times, difficulty in coordinating meeting schedules, and inadequate health management for the elderly. Furthermore, tracking residents' health status and immediate response to emergencies are insufficient, necessitating a comprehensive system to improve community safety and quality of life.

[0105] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0106] In this invention, the server includes means for retrieving the schedule of community activities from a storage device and sending notifications to residents; means for tracking the location information of waste collection devices and providing it to residents immediately; means for automatically adjusting and notifying community group meeting schedules based on the availability of participants; and means for providing health monitoring functions to residents' terminals and notifying them of medical-related information. This makes community activities more efficient, reduces the burden on residents, and enables a rapid response in emergencies.

[0107] "Community activities" refer to various events and activities related to daily life that are carried out jointly by residents and organizations within a local community.

[0108] A "storage device" is a device that collects and stores information as a database and retrieves and uses it as needed.

[0109] "Residents" refers to people who live in a specific area or building, and in this invention, they play the role of information recipients.

[0110] "Waste collection equipment" refers to vehicles or devices used to collect specific types of waste and process them appropriately.

[0111] "Location information" refers to the current geographical location of a specific object or person, and is usually expressed as latitude and longitude coordinates.

[0112] A "local group" refers to a collection of people or organizations that share social relationships within a specific region.

[0113] "Free time" refers to periods in one's schedule when there are no other scheduled events.

[0114] "Health information" refers to data that indicates an individual's physical condition, including body temperature, blood pressure, and pulse rate.

[0115] "Medical information" refers to advice or warnings regarding an individual's health condition or illness, or important notices from healthcare providers.

[0116] "Terminal" generally refers to electronic devices such as smartphones, personal computers, and tablets, and in this invention, it refers to a device that displays information or inputs data.

[0117] The server retrieves the schedule of local activities from a storage device and runs a program that sends notifications to residents' terminals. This program uses MySQL® as its database management system to organize and manage schedule information that differs for each region. Firebase Cloud Messaging is used for communication, ensuring that notifications are sent efficiently.

[0118] Furthermore, the server obtains location information from GPS devices installed on the waste collection equipment and immediately provides that information to residents' devices using the Google® Maps API. This allows residents to monitor in real time how close the waste collection equipment is to their area.

[0119] For scheduling meetings within local groups, the server automatically checks participants' availability and adjusts meeting dates. Schedule information is managed using the Google Calendar API and is designed to ensure timely notifications. Furthermore, the Google Cloud Speech-to-Text API is used for real-time transcription of meeting speeches. This automatically generates meeting minutes, which are then shared with participants.

[0120] In managing health information, residents use their smartphones to input daily health information and send it to a server. This information is analyzed using Python and TENSORFLOW®, and an AI model provides residents with advice compared to standard values. Based on this, residents can understand their own health status and consult with medical institutions as needed.

[0121] As a concrete example, suppose an elderly person uses their smartphone every morning to record their body temperature and blood pressure and sends it to a server. Once the data reaches the server, a Python script is executed, an AI model evaluates any anomalies, and automatically sends advice such as, "Your morning body temperature is higher than normal. Please consider consulting a doctor," if necessary.

[0122] An example of a prompt might be: "Explain how residents can use the health monitoring features provided by the Smart City Assistant to understand their health status in real time and receive necessary medical advice."

[0123] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0124] Step 1:

[0125] The server retrieves schedule information for local activities from a storage device. It queries the schedule information stored in the MySQL database and extracts the notification target data for the specified time. The input is the local ID, and the output is the corresponding schedule data.

[0126] Step 2:

[0127] The server uses Firebase Cloud Messaging to retrieve schedule information and send it to the residents' devices. The extracted schedule data is formatted as a message, and a notification is sent to each resident's registered device. The input is the schedule data, and the output is the notification displayed on the resident's device.

[0128] Step 3:

[0129] The server obtains location information for waste collection equipment. It receives current coordinate data from a GPS device installed on the collection equipment and creates map data for visualization using the Google Maps API. The input is GPS coordinates, and the output is map data.

[0130] Step 4:

[0131] The terminal visually displays the current location of the waste collection equipment to the user based on the displayed map data. Users can check real-time location information on the app. The input is map data, and the output is a map display on the user's terminal.

[0132] Step 5:

[0133] The server retrieves the schedules of meeting participants in a regional group using the Google Calendar API and adjusts the meeting schedule based on their availability. It analyzes the retrieved schedule data, finds matching available time slots, and sets the date. The input is the participants' calendar data, and the output is the optimal meeting date.

[0134] Step 6:

[0135] The server acquires audio data during the meeting and converts it into text in real time using the Google Cloud Speech-to-Text API. The input is audio data, and the output is text data for meeting minutes.

[0136] Step 7:

[0137] The server receives health information sent from the elderly person's device and analyzes it using a generative AI model. It uses Python and TensorFlow to determine if the values ​​are outliers and generates advice. The input is health data, and the output is a generated health advice message.

[0138] Step 8:

[0139] The server sends the generated advice to the resident's device. Firebase Cloud Messaging is used again to execute the process of notifying the resident's device of the health advice. The input is the advice message, and the output is the notification displayed on the device.

[0140] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0141] This invention combines an emotion engine with an AI system that supports community activities and personal health management. This system recognizes the user's emotions and provides more personalized feedback in community activities and health management. Specific embodiments of each function are described below.

[0142] First, the server runs an AI model responsible for emotion recognition. It analyzes user input (e.g., voice, text, and sometimes images) and extracts emotional information from it. It recognizes emotional states such as joy, sadness, and anger as emotional data. This information is used to provide feedback tailored to the current emotional state of the residents.

[0143] For example, if a user is perceived as "frustrated" when receiving a garbage collection notification, the system will simplify and make the notification more straightforward, or add words of encouragement to make it easier for the user to accept. Similarly, in a neighborhood association meeting, if a participant is feeling "anxious," the system will provide a more detailed explanation of the agenda or encourage them to ask questions in advance to ensure the meeting proceeds smoothly.

[0144] Furthermore, the emotion engine tracks users' emotional tendencies over the long term by accumulating and analyzing their emotional history in a database. This data can serve as reference information when adjusting the management policies of local communities. For example, if there are signs of a decline in overall community well-being, it can provide a starting point for suggesting events or new initiatives.

[0145] In the field of elderly care, it is possible to provide special attention to elderly individuals whose feelings of loneliness and stress are emphasized through emotion recognition. For example, if the emotion "loneliness" is detected, the server can send a message recommending communication with family and friends.

[0146] Thus, the system incorporating the emotion engine of the present invention can achieve flexible responses that take into account the user's mental state, thereby improving the quality of community activities. This will improve the quality of life for residents and contribute to the revitalization of the entire community.

[0147] The following describes the processing flow.

[0148] Step 1: The user inputs data into the device regarding their daily activities, health status, willingness to participate in community activities, etc., or speaks about these topics.

[0149] Step 2: The device sends input data or voice data to the server in real time.

[0150] Step 3: Based on the data received by the server, the emotion engine is activated and natural language processing and sentiment analysis are performed.

[0151] Step 4: The server analyzes the user's emotional state using the emotion engine, classifies the analysis results, and saves them in an emotion folder.

[0152] Step 5: The server generates the most appropriate response based on the analysis results, considering the user's state. For example, if the server detects that the user is "frustrated," it provides concise and clear information.

[0153] Step 6: The server sends the generated response to the terminal and provides personalized feedback to the user.

[0154] Step 7: The user reviews the feedback displayed on their device and enters replies or additional data into the system as needed.

[0155] Step 8: The server activates a function that stores long-term emotional data in a database and analyzes emotional trends over time.

[0156] Step 9: The server provides the aggregated emotional data to stakeholders as material for planning community activities and proposing new services.

[0157] Through this series of steps, the present invention can provide services that are sensitive to the feelings of residents and improve the level of engagement of the entire local community.

[0158] (Example 2)

[0159] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0160] In modern society, community activities and health management often fail to address the individual needs of residents, with challenges including inefficient information provision and a lack of mental care. Furthermore, there is difficulty in understanding the overall well-being and trends of the community and implementing appropriate measures based on that understanding. Traditional systems are unable to provide individualized feedback based on residents' emotional states, thus failing to contribute to the revitalization of the community and the improvement of residents' quality of life.

[0161] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0162] This invention includes a server that includes means for retrieving plans for local events from a data storage device and sending notifications to residents, means for tracking and instantly providing residents with information on the placement of garbage collection vehicles, and means for automatically resetting and notifying residents of community meeting schedules based on their availability. This improves the accuracy of information provision regarding the community surrounding residents and enables community revitalization and improvement of residents' quality of life through the use of personalized feedback.

[0163] A "data storage device" is a device used to store information such as plans for local events and to retrieve that data as needed.

[0164] "Resident" refers to an individual who lives or works within a specific area.

[0165] "Means of transmitting information" refers to means used to notify residents of information, including electronic messages and notifications.

[0166] A "waste collection vehicle" is a vehicle used to collect and transport waste within a region.

[0167] "Location information" refers to positional information that indicates where an object or person is currently located.

[0168] "Tracking" is the act of continuously monitoring the movements of a specific object or piece of information.

[0169] "Community meeting schedule" refers to the dates of meetings, events, and other gatherings held within the local community.

[0170] "Reconfiguration" refers to the act of changing existing settings and adjusting them based on new conditions or requirements.

[0171] "Textual information" refers to data that converts audio information into text, and is expressed as text.

[0172] A "document" refers to a collection of textual information compiled about a particular subject.

[0173] "Health information" refers to data related to an individual's health status, and may include body temperature, pulse rate, etc.

[0174] A "reference value" refers to a numerical value or state that is considered normal under specific conditions.

[0175] "Advice" refers to suggestions or suggestions offered in response to a specific situation or problem.

[0176] An "AI model" is a computational model that performs a specific task based on artificial intelligence.

[0177] "Personalized feedback" refers to responses and information tailored to the specific circumstances and needs of a particular user.

[0178] "Emotional history" refers to a record of past emotional states and is used to track a user's emotional tendencies.

[0179] "Means of analyzing trends" refers to methods or tools used to evaluate patterns and trends contained in collected data.

[0180] This invention incorporates an emotion engine into a system designed to support community activities and individual health management. The system aims to improve the quality of life for residents by identifying users' emotions and providing feedback based on that data.

[0181] Hardware and software to be used

[0182] This system uses two main hardware components: a server and a terminal. The server functions as a platform for running a generative AI model that performs emotion recognition. The terminal is responsible for collecting user input and sending it to the server. Data processing uses software libraries for emotion analysis and natural language processing models.

[0183] Data processing and calculations

[0184] The server inputs audio, text, and sometimes image data received from the user via the device into an emotion recognition model and analyzes the emotional information. The recognized emotional data includes a variety of states such as joy, sadness, and anger. The server then uses a generative AI model to generate personalized feedback. This feedback, optimized for the user's current emotional state, is delivered to the user via the device.

[0185] Examples of specific cases and prompt statements

[0186] For example, if the server detects that a user is experiencing stress based on their input, it can provide the user with encouraging messages or suggestions for relaxation. Furthermore, by analyzing the emotional history of the entire local community, it can suggest events to address common problems such as declining well-being.

[0187] An example of a prompt would be, "Please propose a new event to improve well-being in this area." By inputting this prompt into the AI ​​model, specific activity proposals tailored to the needs of the region can be obtained.

[0188] In this way, this system enables flexible responses that adapt to the user's emotional state, contributing to the qualitative improvement and revitalization of local communities.

[0189] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0190] Step 1:

[0191] Users input information into the system through their devices. This input data can be voice, text, or images. For example, a user might speak about their emotions or health status into their smartphone, and this information is recorded on the device.

[0192] Step 2:

[0193] The terminal sends the input data received from the user to the server. At this time, the data is formatted into a format that can be processed by the server. Specifically, voice data is converted into text data by speech recognition software.

[0194] Step 3:

[0195] The server inputs the received data into a generative AI model for emotion recognition, analyzing the user's emotions. Emotional information such as joy, sadness, and anger is extracted. Through this computational process, various emotional states are identified from the data.

[0196] Step 4:

[0197] The server generates personalized feedback based on emotional data. A generative AI model uses specific prompts to create appropriate messages. For example, "In this case, we recommend listening to music to relax."

[0198] Step 5:

[0199] The server generates feedback and notifies the user via their device. The user receives advice tailored to their situation and can use it to guide their actions.

[0200] Step 6:

[0201] The server stores users' emotional history in a database and analyzes long-term emotional trends. This historical data is used to extract information on overall community well-being and problems, serving as a source of information for guiding community improvement proposals.

[0202] (Application Example 2)

[0203] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0204] In community activities and health management for the elderly, there is a lack of support that takes into account the emotional state of individual users. Therefore, flexible and individualized responses are needed to improve the quality of life in the community. Furthermore, reducing feelings of loneliness and stress among the elderly is considered important, and to solve these problems, a new system is needed that recognizes emotions and enables individualized responses.

[0205] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0206] In this invention, the server includes means for obtaining a schedule of local activities from a database and sending notifications to users; means for tracking the location information of waste collection machines and providing it to users immediately; means for converting audio information during meetings into text information and generating meeting minutes; and means for recognizing the emotional state of users and providing personalized support in accordance with those emotions. This enables individualized responses in accordance with users' emotions, as well as the smooth implementation of local activities and improved health management for the elderly.

[0207] A "database" is an information processing system that stores various information such as schedules for community activities and information about users, and allows for searching and referencing as needed.

[0208] "Community activities" refer to various events, gatherings, volunteer activities, and community services within the local community, and include organized actions to promote resident participation and cooperation.

[0209] A "waste collection machine" refers to a machine used to collect waste from a community and transport it to a designated processing facility. Specifically, this includes garbage trucks.

[0210] "Location information" refers to data that indicates the geographical location of a specific object or person, and is usually expressed as coordinates or an address.

[0211] "Auditory information" refers to information that can be acquired through hearing, such as human speech and ambient sounds. It is mainly converted into digital data using recording technology.

[0212] "Textual information" refers to linguistic information that has been digitized using character codes, and takes a form that is easy to communicate and store.

[0213] "Meeting minutes" are documents that record the content of discussions and decisions made at meetings or gatherings, and organize them in a way that allows for later reference.

[0214] "Health information" refers to data about an individual's health status, including physiological data such as vital signs and diagnostic results.

[0215] "Emotional state" refers to an individual's psychological state as a response to a specific situation or stimulus, and includes states such as joy, sadness, anger, and anxiety.

[0216] "Individualized support" refers to special responses or services provided in a way that is most appropriate, taking into account the needs and emotional state of each user.

[0217] In this invention, a server, which is an information processing device, combines various hardware and software to provide services that support community activities and health management. The server uses a database system to obtain the schedule of community activities and sends notifications to the user's terminal. It also uses GPS tracking technology to immediately obtain the location information of waste collection machines and provides it to the user.

[0218] The server uses speech processing software to convert audio information during meetings into text and automatically generates meeting minutes. This text conversion is achieved by utilizing a speech recognition engine. Furthermore, the server is equipped with an emotion recognition system that processes audio and image data acquired through the user's terminal to determine the user's emotional state. This function enables personalized support tailored to emotions, and in particular, it can improve the quality of life for the elderly by providing appropriate advice based on health information.

[0219] This technology can be useful, for example, in suggesting relaxing music and activities to alleviate feelings of loneliness that older adults often experience. Specifically, if the system detects that a user is feeling lonely, it can send notifications suggesting communication with family and friends, or relaxation activities such as sunbathing or taking a walk.

[0220] Furthermore, an example of generating prompt text in these processes using a generative AI model is: "I want to develop a care support application to alleviate feelings of loneliness for the elderly. Please give me advice on writing a program that analyzes the user's emotions in real time and generates suggestions to encourage communication with family members when necessary."

[0221] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0222] Step 1:

[0223] The server retrieves the schedule of local activities from the database and sends a notification to the user's device. It sends a query about local activities to the database as input, and the activity schedule information is obtained as output. Based on this information, a notification is generated in text format and sent to the user's device.

[0224] Step 2:

[0225] The server uses GPS tracking technology to collect location information from waste collection machines and provides this location information to the user's device. It receives real-time location data from waste collection machines as input and outputs location information that can be displayed on a map. This location information is displayed as a graphical interface on the user's device.

[0226] Step 3:

[0227] The server uses audio processing software to acquire audio information during a meeting and convert it into text. It receives audio data from a microphone as input and outputs a text-based meeting transcript. A speech recognition engine analyzes the sound wave data and converts it into text.

[0228] Step 4:

[0229] The server acquires voice and image data from the user and determines the user's emotional state through an emotion recognition system. It receives the user's voice and facial image data as input, and evaluates their emotional state as output. An emotion analysis algorithm processes the data and assigns emotion labels such as joy or sadness.

[0230] Step 5:

[0231] Based on the user's emotional state, the server provides personalized support. It receives the results of an emotional analysis as input and generates optimal suggestions and notifications for the user as output. For example, if the server determines the user is feeling lonely, a message encouraging communication is sent to the device, supporting the user's actions.

[0232] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0233] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0234] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0235] [Second Embodiment]

[0236] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0237] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0238] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0239] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0240] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0241] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0242] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0243] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0244] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0245] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0246] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0247] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0248] This invention provides an AI agent system that supports community activities. This system primarily functions in three areas: garbage management, neighborhood association operation support, and elderly care support.

[0249] First, the server extracts garbage collection schedule information from a local activity database and sends a notification to residents' devices based on this information. Residents' devices receive this notification and display a reminder at the set time. This entire process helps residents put out their garbage on the correct dates and times. For example, if combustible waste is collected on Mondays and Thursdays, the server automatically notifies residents, and their devices send a reminder in the morning.

[0250] Next, the server receives location data from GPS devices installed on garbage trucks and makes it accessible to residents as map information. Residents can then view this information through an application and monitor in real time whether their garbage is being collected properly. For example, when a resident opens the app, they can see that a garbage truck is heading towards their area.

[0251] The neighborhood association's operational support system incorporates a server that checks each participant's availability, automatically adjusts meeting schedules, and sends notifications. Furthermore, during meetings, audio data is automatically transcribed, generating meeting minutes. These minutes are shared with participants, who can review them on their devices and make corrections as needed.

[0252] Finally, as part of support for the elderly, a system has been introduced that allows users to input daily health data using a terminal and send it to a server. The server analyzes this data, compares it to standard values, evaluates the user's health status, and generates advice. For example, if body temperature or blood pressure exceeds a certain range, a health alert is automatically sent.

[0253] This system is designed to streamline local community activities and reduce the burden on residents and stakeholders. In this way, it can help address depopulation in the region and strengthen cooperation among residents.

[0254] The following describes the processing flow.

[0255] Step 1: The server accesses the local activity schedule database to retrieve the latest garbage collection schedule information.

[0256] Step 2: The server uses the residents' contact information to send the retrieved garbage collection schedule as a push notification to each resident's device.

[0257] Step 3: Set up the device to display received notifications and provide reminders to residents at the specified time.

[0258] Step 4: The user checks the notification on their device and prepares to take out the trash if necessary.

[0259] Step 5: The server collects location information in real time from GPS devices installed in the garbage trucks.

[0260] Step 6: The server analyzes the location information it has collected and formats the data to display the route of the collection vehicle on a map.

[0261] Step 7: Users check the current location and planned route of the garbage truck via a smartphone app.

[0262] Step 8: For neighborhood association operation support, the server retrieves participants' calendar information and automatically adjusts the optimal meeting schedule.

[0263] Step 9: The server notifies participants of the meeting date as a calendar entry on their devices.

[0264] Step 10: The user checks the meeting schedule on their device and makes manual changes if necessary.

[0265] Step 11: During the meeting, the user collects audio data using a recording device.

[0266] Step 12: The server receives the recorded audio data and converts it into text data using speech recognition technology.

[0267] Step 13: The server checks the converted text data and automatically generates the first version of the meeting minutes.

[0268] Step 14: The user reviews the initial version of the meeting minutes, makes any necessary revisions, and finalizes the official version.

[0269] Step 15: Every morning, the elderly person uses a device to input health data (e.g., body temperature, blood pressure) and sends it to the server.

[0270] Step 16: The server analyzes the received health data and evaluates the health status by comparing it to standard health values.

[0271] Step 17: Based on the evaluation results, the server generates necessary advice and notifies the user's terminal.

[0272] Step 18: Review the advice the user received and use it to help manage their health.

[0273] (Example 1)

[0274] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0275] In local communities, there is a need for systems that efficiently support diverse activities in which residents participate and the health management of the elderly. However, existing methods suffer from delays in information dissemination, difficulties in coordination, and limitations in providing individualized support. Therefore, efficiently managing information related to community activities and providing appropriate support to each resident is a challenge.

[0276] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0277] In this invention, the server includes means for acquiring scheduled information on local activities from an information storage means via a communication medium and transmitting notifications to a receiving device; means for collecting location information of mobile objects and providing it to residents immediately; and means for automatically adjusting and notifying participants of the meeting date and time of local organizations based on their available time. This enables the rapid provision of information on local activities, the smooth coordination of various events, and the efficient provision of individual support.

[0278] "Communication medium" refers to the means and channels for transmitting and receiving information, and is a general term that includes wired and wireless technologies.

[0279] "Information storage means" refers to the devices and systems used to store and manage data, such as databases.

[0280] "Receiver device" refers to the equipment for receiving information transmitted from a server, including smartphones, tablets, etc.

[0281] "Location information of a moving object" refers to the data indicating the geographical location of a specific object or vehicle, which is obtained by technologies such as GPS.

[0282] "Local community" refers to the collective body or organization of residents in a specific geographical area, such as a neighborhood association or a local government council.

[0283] "Collective date and time" is a term that refers to the date and time when a specific activity or event is carried out.

[0284] "Available time" is the information indicating the date and time when an individual or group can participate in a specific activity.

[0285] "Notification means" refers to the methods and technologies used to transmit information to the recipient, such as email and push notifications.

[0286] This invention is a system for efficiently supporting local activities, mainly composed of a server, a terminal, and a user. The server accesses the information storage means specialized for the region and obtains the schedule information of local activities. This information is mainly managed using a database management system. The server transmits the acquired information as a notification to the receiver device via a communication medium. The notification is provided via a protocol (e.g., HTTP) using the Internet and a communication service. As a specific example, it is conceivable to send a reminder of the neighborhood association's event to the residents' terminals every Monday.

[0287] Furthermore, the server collects the location information of the mobile object and provides this information to the residents in real time. For this, data processing using a GPS device or a map API is performed. The terminal can visually present this information to the residents through an application. For example, when a resident opens the app, the current location of the garbage collection truck is displayed on the map.

[0288] Also, in order to support the efficient operation of meetings of local organizations, the server collects the free time of each participant and automatically adjusts the optimal meeting date and time. For this, the calendar API is utilized to aggregate individual schedule information and perform calculations to derive an optimal solution. During the meeting, voice information is acquired using a recording device and converted into character information by applying voice recognition technology. The character information generated as the minutes of the meeting is shared with the user's terminal and confirmed and corrected at a later date.

[0289] Finally, health information is input from the elderly via the terminal and transmitted to the server. The health information is compared with standard values by a data analysis program within the server, and the health status is evaluated. The evaluation results and necessary advice are notified to the user's terminal. As a result, for example, an alert such as "Your body temperature is high today, so please rest" may be sent to the elderly.

[0290] An example of the prompt sentence is "Please briefly explain the process of automatically adjusting the meeting schedule of the neighborhood association." This invention provides an effective means for comprehensively supporting the lives of residents and smoothly promoting local activities.

[0291] The flow of the specific process in Example 1 will be described using FIG. 11.

[0292] Step 1:

[0293] The server accesses a database of local activities and extracts schedule information relevant to residents. It uses tables and query conditions within the database as input and retrieves the schedule information in JSON format as output. Based on this information, the server prepares notifications and organizes the data for sending them via communication media. Specifically, it extracts information such as "The next neighborhood association meeting is next Wednesday" from the database.

[0294] Step 2:

[0295] The device receives notifications sent from the server and displays reminders at the appropriate time based on the user's schedule. It receives notification data from the server as input and displays reminders on the user's screen as output. Specifically, a pop-up message appears on the device saying, "There is a neighborhood association meeting tomorrow."

[0296] Step 3:

[0297] The server receives current location information from the garbage truck's GPS device. It takes location data from the GPS as input and updates that data as map information in real time as output. The server uses a map API to generate visual location information and provides it to the terminal. Specifically, it displays "Garbage truck passing through area A" on the map.

[0298] Step 4:

[0299] The device displays map information provided by the server, allowing the user to check the location of the garbage truck in real time. It receives map data from the server as input and displays routes and location icons on the map application as output. In practice, the app displays updated information such as "10 minutes until the garbage truck arrives."

[0300] Step 5:

[0301] The server refers to the calendars of meeting participants and calculates the optimal meeting schedule. As input, it obtains the calendar information (list of available time slots) of the participants, and as output, it generates a new adjusted meeting schedule. As a specific operation, information such as "The next meeting is set at 3 pm on Friday" is calculated.

[0302] Step 6:

[0303] The terminal receives the updated meeting schedule from the server and notifies the participants. As input, it receives the new meeting schedule, and as output, it adds it as an event in the calendar. In a specific operation, a notification saying "A new meeting schedule has been added" reaches the calendar app on the terminal.

[0304] Step 7:

[0305] The user uses the terminal to input their health information and send it to the server. As input, there are health indicators such as body temperature and blood pressure, and as output, health data is sent to the server. As a specific operation, input "36.7 degrees" for body temperature in the app and tap the send button.

[0306] Step 8:

[0307] The server analyzes the received health data, compares it with the standard values, and evaluates the health status. Using the health data sent from the user as input, it generates analysis results and advice as output. As a specific operation, it creates a message such as "Your blood pressure is on the high side, so drink more water".

[0308] Step 9:

[0309] The terminal receives the health analysis results from the server and displays an alert to the user. Receiving the advice message from the server as input, a warning is directly shown on the user screen as output. As a specific operation, "Health warning: Pay attention as your body temperature is high today" is displayed in the notification area of the app.

[0310] (Application Example 1)

[0311] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0312] To efficiently manage and promote community activities, the rapid provision of various information and smooth communication are essential. However, conventional methods have problems such as missing garbage collection times, difficulty in coordinating meeting schedules, and inadequate health management for the elderly. Furthermore, tracking residents' health status and immediate response to emergencies are insufficient, necessitating a comprehensive system to improve community safety and quality of life.

[0313] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0314] In this invention, the server includes means for retrieving the schedule of community activities from a storage device and sending notifications to residents; means for tracking the location information of waste collection devices and providing it to residents immediately; means for automatically adjusting and notifying community group meeting schedules based on the availability of participants; and means for providing health monitoring functions to residents' terminals and notifying them of medical-related information. This makes community activities more efficient, reduces the burden on residents, and enables a rapid response in emergencies.

[0315] "Community activities" refer to various events and activities related to daily life that are carried out jointly by residents and organizations within a local community.

[0316] A "storage device" is a device that collects and stores information as a database and retrieves and uses it as needed.

[0317] "Residents" refers to people who live in a specific area or building, and in this invention, they play the role of information recipients.

[0318] "Waste collection equipment" refers to vehicles or devices used to collect specific types of waste and process them appropriately.

[0319] "Location information" refers to the current geographical location of a specific object or person, and is usually expressed as latitude and longitude coordinates.

[0320] A "local group" refers to a collection of people or organizations that share social relationships within a specific region.

[0321] "Free time" refers to periods in one's schedule when there are no other scheduled events.

[0322] "Health information" refers to data that indicates an individual's physical condition, including body temperature, blood pressure, and pulse rate.

[0323] "Medical information" refers to advice or warnings regarding an individual's health condition or illness, or important notices from healthcare providers.

[0324] "Terminal" generally refers to electronic devices such as smartphones, personal computers, and tablets, and in this invention, it refers to a device that displays information or inputs data.

[0325] The server retrieves the schedule of local activities from a storage device and runs a program that sends notifications to residents' terminals. This program uses MySQL as its database management system to organize and manage schedule information that differs for each region. Firebase Cloud Messaging is used for communication, ensuring that notifications are sent efficiently.

[0326] Furthermore, the server obtains location information from GPS devices installed on the waste collection equipment and immediately provides that information to residents' devices using the Google Maps API. This allows residents to monitor in real time how close the waste collection equipment is to their area.

[0327] For scheduling meetings within local groups, the server automatically checks participants' availability and adjusts meeting dates. Schedule information is managed using the Google Calendar API and is designed to ensure timely notifications. Furthermore, the Google Cloud Speech-to-Text API is used for real-time transcription of meeting speeches. This automatically generates meeting minutes, which are then shared with participants.

[0328] In managing health information, residents use their smartphones to input daily health data and send it to a server. This information is analyzed using Python and TensorFlow, and an AI model provides residents with advice compared to standard values. Based on this, residents can understand their own health status and consult with medical institutions as needed.

[0329] As a concrete example, suppose an elderly person uses their smartphone every morning to record their body temperature and blood pressure and sends it to a server. Once the data reaches the server, a Python script is executed, an AI model evaluates any anomalies, and automatically sends advice such as, "Your morning body temperature is higher than normal. Please consider consulting a doctor," if necessary.

[0330] An example of a prompt might be: "Explain how residents can use the health monitoring features provided by the Smart City Assistant to understand their health status in real time and receive necessary medical advice."

[0331] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0332] Step 1:

[0333] The server retrieves schedule information for local activities from a storage device. It queries the schedule information stored in the MySQL database and extracts the notification target data for the specified time. The input is the local ID, and the output is the corresponding schedule data.

[0334] Step 2:

[0335] The server uses Firebase Cloud Messaging to retrieve schedule information and send it to the residents' devices. The extracted schedule data is formatted as a message and sent as a notification to each resident's registered device. The input is the schedule data, and the output is the notification displayed on the resident's device.

[0336] Step 3:

[0337] The server obtains location information for waste collection equipment. It receives current coordinate data from a GPS device installed on the collection equipment and creates map data for visualization using the Google Maps API. The input is GPS coordinates, and the output is map data.

[0338] Step 4:

[0339] The terminal visually displays the current location of the waste collection equipment to the user based on the displayed map data. Users can check real-time location information on the app. The input is map data, and the output is a map display on the user's terminal.

[0340] Step 5:

[0341] The server retrieves the schedules of meeting participants in a regional group using the Google Calendar API and adjusts the meeting schedule based on their availability. It analyzes the retrieved schedule data, finds matching available time slots, and sets the date. The input is the participants' calendar data, and the output is the optimal meeting date.

[0342] Step 6:

[0343] The server acquires audio data during the meeting and converts it into text in real time using the Google Cloud Speech-to-Text API. The input is audio data, and the output is text data for meeting minutes.

[0344] Step 7:

[0345] The server receives health information sent from the elderly person's device and analyzes it using a generative AI model. It uses Python and TensorFlow to determine if the values ​​are outliers and generates advice. The input is health data, and the output is a generated health advice message.

[0346] Step 8:

[0347] The server sends the generated advice to the resident's device. Firebase Cloud Messaging is used again to execute the process of notifying the resident's device of the health advice. The input is the advice message, and the output is the notification displayed on the device.

[0348] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0349] This invention combines an emotion engine with an AI system that supports community activities and personal health management. This system recognizes the user's emotions and provides more personalized feedback in community activities and health management. Specific embodiments of each function are described below.

[0350] First, the server runs an AI model responsible for emotion recognition. It analyzes user input (e.g., voice, text, and sometimes images) and extracts emotional information from it. It recognizes emotional states such as joy, sadness, and anger as emotional data. This information is used to provide feedback tailored to the current emotional state of the residents.

[0351] For example, if a user is perceived as "frustrated" when receiving a garbage collection notification, the system will simplify and make the notification more straightforward, or add words of encouragement to make it easier for the user to accept. Similarly, in a neighborhood association meeting, if a participant is feeling "anxious," the system will provide a more detailed explanation of the agenda or encourage them to ask questions in advance to ensure the meeting proceeds smoothly.

[0352] Furthermore, the emotion engine tracks users' emotional tendencies over the long term by accumulating and analyzing their emotional history in a database. This data can serve as reference information when adjusting the management policies of local communities. For example, if there are signs of a decline in overall community well-being, it can provide a starting point for suggesting events or new initiatives.

[0353] In the field of elderly care, it is possible to provide special attention to elderly individuals whose feelings of loneliness and stress are emphasized through emotion recognition. For example, if the emotion "loneliness" is detected, the server can send a message recommending communication with family and friends.

[0354] Thus, the system incorporating the emotion engine of the present invention can achieve flexible responses that take into account the user's mental state, thereby improving the quality of community activities. This will improve the quality of life for residents and contribute to the revitalization of the entire community.

[0355] The following describes the processing flow.

[0356] Step 1: The user inputs data into the device regarding their daily activities, health status, willingness to participate in community activities, etc., or speaks about these topics.

[0357] Step 2: The device sends input data or voice data to the server in real time.

[0358] Step 3: Based on the data received by the server, the emotion engine is activated and natural language processing and sentiment analysis are performed.

[0359] Step 4: The server analyzes the user's emotional state using the emotion engine, classifies the analysis results, and saves them in an emotion folder.

[0360] Step 5: The server generates the most appropriate response based on the analysis results, considering the user's state. For example, if the server detects that the user is "frustrated," it provides concise and clear information.

[0361] Step 6: The server sends the generated response to the terminal and provides personalized feedback to the user.

[0362] Step 7: The user reviews the feedback displayed on their device and enters replies or additional data into the system as needed.

[0363] Step 8: The server activates a function that stores long-term emotional data in a database and analyzes emotional trends over time.

[0364] Step 9: The server provides the aggregated emotional data to stakeholders as material for planning community activities and proposing new services.

[0365] Through this series of steps, the present invention can provide services that are sensitive to the feelings of residents and improve the level of engagement of the entire local community.

[0366] (Example 2)

[0367] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0368] In modern society, community activities and health management often fail to address the individual needs of residents, with challenges including inefficient information provision and a lack of mental care. Furthermore, there is difficulty in understanding the overall well-being and trends of the community and implementing appropriate measures based on that understanding. Traditional systems are unable to provide individualized feedback based on residents' emotional states, thus failing to contribute to the revitalization of the community and the improvement of residents' quality of life.

[0369] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0370] This invention includes a server that includes means for retrieving plans for local events from a data storage device and sending notifications to residents, means for tracking and instantly providing residents with information on the placement of garbage collection vehicles, and means for automatically resetting and notifying residents of community meeting schedules based on their availability. This improves the accuracy of information provision regarding the community surrounding residents and enables community revitalization and improvement of residents' quality of life through the use of personalized feedback.

[0371] A "data storage device" is a device used to store information such as plans for local events and to retrieve that data as needed.

[0372] "Resident" refers to an individual who lives or works within a specific area.

[0373] "Means of transmitting information" refers to means used to notify residents of information, including electronic messages and notifications.

[0374] A "waste collection vehicle" is a vehicle used to collect and transport waste within a region.

[0375] "Location information" refers to positional information that indicates where objects or people are currently located.

[0376] "Tracking" is the act of continuously monitoring the movements of a specific object or piece of information.

[0377] "Community meeting schedule" refers to the dates of meetings, events, and other gatherings held within the local community.

[0378] "Reconfiguration" refers to the act of changing existing settings and adjusting them based on new conditions or requirements.

[0379] "Textual information" refers to data that converts audio information into text, and is expressed as text.

[0380] A "document" refers to a collection of textual information compiled about a particular subject.

[0381] "Health information" refers to data related to an individual's health status, and may include body temperature, pulse rate, etc.

[0382] A "reference value" refers to a numerical value or state that is considered normal under specific conditions.

[0383] "Advice" refers to suggestions or suggestions offered in response to a specific situation or problem.

[0384] An "AI model" is a computational model that performs a specific task based on artificial intelligence.

[0385] "Personalized feedback" refers to responses and information tailored to the specific circumstances and needs of a particular user.

[0386] "Emotional history" refers to a record of past emotional states and is used to track a user's emotional tendencies.

[0387] "Means of analyzing trends" refers to methods or tools used to evaluate patterns and trends contained in collected data.

[0388] This invention incorporates an emotion engine into a system designed to support community activities and individual health management. The system aims to improve the quality of life for residents by identifying users' emotions and providing feedback based on that data.

[0389] Hardware and software to be used

[0390] This system uses two main hardware components: a server and a terminal. The server functions as a platform for running a generative AI model that performs emotion recognition. The terminal is responsible for collecting user input and sending it to the server. Data processing uses software libraries for emotion analysis and natural language processing models.

[0391] Data processing and calculations

[0392] The server inputs audio, text, and sometimes image data received from the user via the device into an emotion recognition model and analyzes the emotional information. The recognized emotional data includes a variety of states such as joy, sadness, and anger. The server then uses a generative AI model to generate personalized feedback. This feedback, optimized for the user's current emotional state, is delivered to the user via the device.

[0393] Examples of specific cases and prompt statements

[0394] For example, if the server detects that a user is experiencing stress based on their input, it can provide the user with encouraging messages or suggestions for relaxation. Furthermore, by analyzing the emotional history of the entire local community, it can suggest events to address common problems such as declining well-being.

[0395] An example of a prompt would be, "Please propose a new event to improve well-being in this area." By inputting this prompt into the AI ​​model, specific activity proposals tailored to the needs of the region can be obtained.

[0396] In this way, this system enables flexible responses that adapt to the user's emotional state, contributing to the qualitative improvement and revitalization of local communities.

[0397] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0398] Step 1:

[0399] Users input information into the system through their devices. This input data can be voice, text, or images. For example, a user might speak about their emotions or health status into their smartphone, and this information is recorded on the device.

[0400] Step 2:

[0401] The terminal sends the input data received from the user to the server. At this time, the data is formatted into a format that can be processed by the server. Specifically, voice data is converted into text data by speech recognition software.

[0402] Step 3:

[0403] The server inputs the received data into a generative AI model for emotion recognition, which then analyzes the user's emotions. Emotional information such as joy, sadness, and anger is extracted. Through this computational process, various emotional states are identified from the data.

[0404] Step 4:

[0405] The server generates personalized feedback based on emotional data. A generative AI model uses specific prompts to create appropriate messages. For example, "In this case, we recommend listening to music to relax."

[0406] Step 5:

[0407] The server generates feedback and notifies the user via their device. The user receives advice tailored to their situation and can use it to guide their actions.

[0408] Step 6:

[0409] The server stores users' emotional history in a database and analyzes long-term emotional trends. This historical data is used to extract information on overall community well-being and problems, serving as a source of information for guiding community improvement proposals.

[0410] (Application Example 2)

[0411] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0412] In community activities and health management for the elderly, there is a lack of support that takes into account the emotional state of individual users. Therefore, flexible and individualized responses are needed to improve the quality of life in the community. Furthermore, reducing feelings of loneliness and stress among the elderly is considered important, and to solve these problems, a new system is needed that recognizes emotions and enables individualized responses.

[0413] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0414] In this invention, the server includes means for obtaining a schedule of local activities from a database and sending notifications to users; means for tracking the location information of waste collection machines and providing it to users immediately; means for converting audio information during meetings into text information and generating meeting minutes; and means for recognizing the emotional state of users and providing personalized support in accordance with those emotions. This enables individualized responses in accordance with users' emotions, as well as the smooth implementation of local activities and improved health management for the elderly.

[0415] A "database" is an information processing system that stores various information such as schedules for community activities and information about users, and allows for searching and referencing as needed.

[0416] "Community activities" refer to various events, gatherings, volunteer activities, and community services within the local community, and include organized actions to promote resident participation and cooperation.

[0417] A "waste collection machine" refers to a machine used to collect waste from a community and transport it to a designated processing facility. Specifically, this includes garbage trucks.

[0418] "Location information" refers to data that indicates the geographical location of a specific object or person, and is usually expressed as coordinates or an address.

[0419] "Auditory information" refers to information that can be acquired through hearing, such as human speech and ambient sounds. It is mainly converted into digital data using recording technology.

[0420] "Textual information" refers to linguistic information digitized using character codes, and takes a form that is easy to communicate and store.

[0421] "Meeting minutes" are documents that record the content of discussions and decisions made at meetings or gatherings, and organize them in a way that allows for later reference.

[0422] "Health information" refers to data about an individual's health status, including physiological data such as vital signs and diagnostic results.

[0423] "Emotional state" refers to an individual's psychological state as a response to a specific situation or stimulus, and includes states such as joy, sadness, anger, and anxiety.

[0424] "Individualized support" refers to special responses or services provided in a way that is most appropriate, taking into account the needs and emotional state of each user.

[0425] In this invention, a server, which is an information processing device, combines various hardware and software to provide services that support community activities and health management. The server uses a database system to obtain the schedule of community activities and sends notifications to the user's terminal. It also uses GPS tracking technology to immediately obtain the location information of waste collection machines and provides it to the user.

[0426] The server uses speech processing software to convert audio information during meetings into text and automatically generates meeting minutes. This text conversion is achieved by utilizing a speech recognition engine. Furthermore, the server is equipped with an emotion recognition system that processes audio and image data acquired through the user's terminal to determine the user's emotional state. This function enables personalized support tailored to emotions, and in particular, it can improve the quality of life for the elderly by providing appropriate advice based on health information.

[0427] This technology can be useful, for example, in suggesting relaxing music and activities to alleviate feelings of loneliness that older adults often experience. Specifically, if the system detects that a user is feeling lonely, it can send notifications suggesting communication with family and friends, or relaxation activities such as sunbathing or taking a walk.

[0428] Furthermore, an example of generating prompt text in these processes using a generative AI model is: "I want to develop a care support application to alleviate feelings of loneliness for the elderly. Please give me advice on writing a program that analyzes the user's emotions in real time and generates suggestions to encourage communication with family members when necessary."

[0429] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0430] Step 1:

[0431] The server retrieves the schedule of local activities from the database and sends a notification to the user's device. It sends a query about local activities to the database as input, and the activity schedule information is obtained as output. Based on this information, a notification is generated in text format and sent to the user's device.

[0432] Step 2:

[0433] The server uses GPS tracking technology to collect location information from waste collection machines and provides this location information to the user's device. It receives real-time location data from waste collection machines as input and outputs location information that can be displayed on a map. This location information is displayed as a graphical interface on the user's device.

[0434] Step 3:

[0435] The server uses audio processing software to acquire audio information during a meeting and converts it into text. It receives audio data from a microphone as input and outputs a text-based meeting transcript. A speech recognition engine analyzes the sound wave data and converts it into text.

[0436] Step 4:

[0437] The server acquires voice and image data from the user and determines the user's emotional state through an emotion recognition system. It receives the user's voice and facial image data as input, and evaluates their emotional state as output. An emotion analysis algorithm processes the data and assigns emotion labels such as joy or sadness.

[0438] Step 5:

[0439] Based on the user's emotional state, the server provides personalized support. It receives the results of an emotional analysis as input and generates optimal suggestions and notifications for the user as output. For example, if the server determines the user is feeling lonely, a message encouraging communication is sent to the device, supporting the user's actions.

[0440] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0441] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0442] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0443] [Third Embodiment]

[0444] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0445] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0446] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0447] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0448] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0449] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0450] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0451] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0452] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0453] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0454] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0455] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0456] This invention provides an AI agent system that supports community activities. This system primarily functions in three areas: garbage management, neighborhood association operation support, and elderly care support.

[0457] First, the server extracts garbage collection schedule information from a local activity database and sends a notification to residents' devices based on this information. Residents' devices receive this notification and display a reminder at the set time. This entire process helps residents put out their garbage on the correct dates and times. For example, if combustible waste is collected on Mondays and Thursdays, the server automatically notifies residents, and their devices send a reminder in the morning.

[0458] Next, the server receives location data from GPS devices installed on garbage trucks and makes it accessible to residents as map information. Residents can then view this information through an application and monitor in real time whether their garbage is being collected properly. For example, when a resident opens the app, they can see that a garbage truck is heading towards their area.

[0459] The neighborhood association's operational support system incorporates a server that checks each participant's availability, automatically adjusts meeting schedules, and sends notifications. Furthermore, during meetings, audio data is automatically transcribed, generating meeting minutes. These minutes are shared with participants, who can review them on their devices and make corrections as needed.

[0460] Finally, as part of support for the elderly, a system has been introduced that allows users to input daily health data using a terminal and send it to a server. The server analyzes this data, compares it to standard values, evaluates the user's health status, and generates advice. For example, if body temperature or blood pressure exceeds a certain range, a health alert is automatically sent.

[0461] This system is designed to streamline local community activities and reduce the burden on residents and stakeholders. In this way, it can help address depopulation in the region and strengthen cooperation among residents.

[0462] The following describes the processing flow.

[0463] Step 1: The server accesses the local activity schedule database to retrieve the latest garbage collection schedule information.

[0464] Step 2: The server uses the residents' contact information to send the retrieved garbage collection schedule as a push notification to each resident's device.

[0465] Step 3: Set up the device to display received notifications and provide reminders to residents at the specified time.

[0466] Step 4: The user checks the notification on their device and prepares to take out the trash if necessary.

[0467] Step 5: The server collects location information in real time from GPS devices installed in the garbage trucks.

[0468] Step 6: The server analyzes the location information it has collected and formats the data to display the route of the collection vehicle on a map.

[0469] Step 7: Users check the current location and planned route of the garbage truck via a smartphone app.

[0470] Step 8: For neighborhood association operation support, the server retrieves participants' calendar information and automatically adjusts the optimal meeting schedule.

[0471] Step 9: The server notifies participants of the meeting date as a calendar entry on their devices.

[0472] Step 10: The user checks the meeting schedule on their device and makes manual changes if necessary.

[0473] Step 11: During the meeting, the user collects audio data using a recording device.

[0474] Step 12: The server receives the recorded audio data and converts it into text data using speech recognition technology.

[0475] Step 13: The server checks the converted text data and automatically generates the first version of the meeting minutes.

[0476] Step 14: The user reviews the initial version of the meeting minutes, makes any necessary revisions, and finalizes the official version.

[0477] Step 15: Every morning, the elderly person uses a device to input health data (e.g., body temperature, blood pressure) and sends it to the server.

[0478] Step 16: The server analyzes the received health data and evaluates the health status by comparing it to standard health values.

[0479] Step 17: Based on the evaluation results, the server generates necessary advice and notifies the user's terminal.

[0480] Step 18: Review the advice the user received and use it to help manage their health.

[0481] (Example 1)

[0482] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0483] In local communities, there is a need for systems that efficiently support diverse activities in which residents participate and the health management of the elderly. However, existing methods suffer from delays in information dissemination, difficulties in coordination, and limitations in providing individualized support. Therefore, efficiently managing information related to community activities and providing appropriate support to each resident is a challenge.

[0484] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0485] In this invention, the server includes means for acquiring scheduled information on local activities from an information storage means via a communication medium and transmitting notifications to a receiving device; means for collecting location information of mobile objects and providing it to residents immediately; and means for automatically adjusting and notifying participants of the meeting date and time of local organizations based on their available time. This enables the rapid provision of information on local activities, the smooth coordination of various events, and the efficient provision of individual support.

[0486] "Communication medium" refers to the means and routes for sending and receiving information, and is a general term that includes wired and wireless technologies.

[0487] "Information storage means" refers to devices and systems used to store and manage data, such as databases.

[0488] A "receiving device" refers to a device used to receive information transmitted from a server, and includes smartphones and tablets.

[0489] "Location information of a moving object" refers to data indicating the geographical location of a specific object or vehicle, and is obtained using technologies such as GPS.

[0490] A "community group" refers to a collection or organization of residents in a specific geographical area, such as a neighborhood association or a local residents' association.

[0491] "Meeting date and time" is a term that refers to the date and time when a particular activity or event takes place.

[0492] "Available time" refers to information indicating the dates and times when an individual or group can participate in a particular activity.

[0493] "Means of notification" refers to the methods and technologies used to transmit information to recipients, such as email and push notifications.

[0494] This invention is a system for efficiently supporting community activities, and mainly consists of a server, terminals, and users. The server accesses a region-specific information storage means and obtains scheduled information on community activities. This information is mainly managed using a database management system. The server transmits the obtained information as a notification to a receiving device via a communication medium. The notification is provided via an internet-based protocol (e.g., HTTP) and communication service. As a specific example, it is conceivable that a reminder of neighborhood association events would be sent to residents' terminals every Monday.

[0495] Furthermore, the server collects location information of moving objects and provides this information to residents in real time. This involves data processing using GPS devices and map APIs. The terminal can then visually present this information to residents through an application. For example, when a resident opens the app, the current location of the garbage truck is displayed on a map.

[0496] Furthermore, to support the efficient management of local group meetings, the server collects each participant's availability and automatically adjusts the optimal meeting time. This utilizes a calendar API to aggregate individual schedule information and perform calculations to derive the optimal solution. During the meeting, audio information is acquired using a recording device and converted into text using speech recognition technology. The generated text information, which serves as meeting minutes, is shared with users' devices for later review and correction.

[0497] Finally, health information is entered by the elderly person via their device and sent to the server. Within the server, a data analysis program compares the health information to standard values ​​and assesses their health status. The assessment results and any necessary advice are then notified to the user's device. This could potentially result in an alert being sent to the elderly person, such as, "Your body temperature is high today, please rest."

[0498] An example of a prompt message is, "Please briefly explain the process for automatically scheduling neighborhood association meetings." This invention provides an effective means to comprehensively support the lives of residents and facilitate community activities.

[0499] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0500] Step 1:

[0501] The server accesses a database of local activities and extracts schedule information relevant to residents. It uses tables and query conditions within the database as input and retrieves the schedule information in JSON format as output. Based on this information, the server prepares notifications and organizes the data for sending them via communication media. Specifically, it extracts information such as "The next neighborhood association meeting is next Wednesday" from the database.

[0502] Step 2:

[0503] The device receives notifications sent from the server and displays reminders at the appropriate time based on the user's schedule. It receives notification data from the server as input and displays reminders on the user's screen as output. Specifically, a pop-up message appears on the device saying, "There is a neighborhood association meeting tomorrow."

[0504] Step 3:

[0505] The server receives current location information from the garbage truck's GPS device. It takes location data from the GPS as input and updates that data as map information in real time as output. The server uses a map API to generate visual location information and provides it to the terminal. Specifically, it displays "Garbage truck passing through area A" on the map.

[0506] Step 4:

[0507] The device displays map information provided by the server, allowing the user to check the location of the garbage truck in real time. It receives map data from the server as input and displays routes and location icons on the map application as output. In practice, the app displays updated information such as "10 minutes until the garbage truck arrives."

[0508] Step 5:

[0509] The server references the calendars of meeting participants and calculates the optimal meeting date. It takes participants' calendar information (a list of their free time slots) as input and generates a new, adjusted meeting date as output. Specifically, it calculates information such as, "The next meeting is scheduled for Friday at 3 PM."

[0510] Step 6:

[0511] The device receives updated meeting schedules from the server and notifies participants. It takes the new meeting schedule as input and adds it as a calendar event as output. Specifically, a notification appears in the device's calendar app stating, "A new meeting has been added."

[0512] Step 7:

[0513] The user uses their device to input their health information and sends it to the server. Inputs include health indicators such as body temperature and blood pressure, and output is health data sent to the server. For example, the user might enter "36.7 degrees" into the app and tap the send button.

[0514] Step 8:

[0515] The server analyzes the received health data and evaluates the user's health status by comparing it to standard values. It uses health data submitted by the user as input and generates analysis results and advice as output. For example, it might create a message such as, "Your blood pressure is a little high, so please drink more water."

[0516] Step 9:

[0517] The device receives health analysis results from the server and displays an alert to the user. It receives advice messages from the server as input and displays a warning directly on the user's screen as output. Specifically, the app's notification area displays "Health Warning: Your body temperature is high today, please be careful."

[0518] (Application Example 1)

[0519] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0520] To efficiently manage and promote community activities, the rapid provision of various information and smooth communication are essential. However, conventional methods have problems such as missing garbage collection times, difficulty in coordinating meeting schedules, and inadequate health management for the elderly. Furthermore, tracking residents' health status and immediate response to emergencies are insufficient, necessitating a comprehensive system to improve community safety and quality of life.

[0521] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0522] In this invention, the server includes means for retrieving the schedule of community activities from a storage device and sending notifications to residents; means for tracking the location information of waste collection devices and providing it to residents immediately; means for automatically adjusting and notifying community group meeting schedules based on the availability of participants; and means for providing health monitoring functions to residents' terminals and notifying them of medical-related information. This makes community activities more efficient, reduces the burden on residents, and enables a rapid response in emergencies.

[0523] "Community activities" refer to various events and activities related to daily life that are carried out jointly by residents and organizations within a local community.

[0524] A "storage device" is a device that collects and stores information as a database and retrieves and uses it as needed.

[0525] "Residents" refers to people who live in a specific area or building, and in this invention, they play the role of information recipients.

[0526] "Waste collection equipment" refers to vehicles or devices used to collect specific types of waste and process them appropriately.

[0527] "Location information" refers to the current geographical location of a specific object or person, and is usually expressed as latitude and longitude coordinates.

[0528] A "local group" refers to a collection of people or organizations that share social relationships within a specific region.

[0529] "Free time" refers to periods in one's schedule when there are no other scheduled events.

[0530] "Health information" refers to data that indicates an individual's physical condition, including body temperature, blood pressure, and pulse rate.

[0531] "Medical information" refers to advice or warnings regarding an individual's health condition or illness, or important notices from healthcare providers.

[0532] "Terminal" generally refers to electronic devices such as smartphones, personal computers, and tablets, and in this invention, it refers to a device that displays information or inputs data.

[0533] The server retrieves the schedule of local activities from a storage device and runs a program that sends notifications to residents' terminals. This program uses MySQL as its database management system to organize and manage schedule information that differs for each region. Firebase Cloud Messaging is used for communication, ensuring that notifications are sent efficiently.

[0534] Furthermore, the server obtains location information from GPS devices installed on the waste collection equipment and immediately provides that information to residents' devices using the Google Maps API. This allows residents to monitor in real time how close the waste collection equipment is to their area.

[0535] For scheduling meetings within local groups, the server automatically checks participants' availability and adjusts meeting dates. Schedule information is managed using the Google Calendar API and is designed to ensure timely notifications. Furthermore, the Google Cloud Speech-to-Text API is used for real-time transcription of meeting speeches. This automatically generates meeting minutes, which are then shared with participants.

[0536] In managing health information, residents use their smartphones to input daily health data and send it to a server. This information is analyzed using Python and TensorFlow, and an AI model provides residents with advice compared to standard values. Based on this, residents can understand their own health status and consult with medical institutions as needed.

[0537] As a concrete example, suppose an elderly person uses their smartphone every morning to record their body temperature and blood pressure and sends it to a server. Once the data reaches the server, a Python script is executed, an AI model evaluates any anomalies, and automatically sends advice such as, "Your morning body temperature is higher than normal. Please consider consulting a doctor," if necessary.

[0538] An example of a prompt might be: "Explain how residents can use the health monitoring features provided by the Smart City Assistant to understand their health status in real time and receive necessary medical advice."

[0539] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0540] Step 1:

[0541] The server retrieves schedule information for local activities from a storage device. It queries the schedule information stored in the MySQL database and extracts the notification target data for the specified time. The input is the local ID, and the output is the corresponding schedule data.

[0542] Step 2:

[0543] The server uses Firebase Cloud Messaging to retrieve schedule information and send it to the residents' devices. The extracted schedule data is formatted as a message and sent as a notification to each resident's registered device. The input is the schedule data, and the output is the notification displayed on the resident's device.

[0544] Step 3:

[0545] The server obtains location information for waste collection equipment. It receives current coordinate data from a GPS device installed on the collection equipment and creates map data for visualization using the Google Maps API. The input is GPS coordinates, and the output is map data.

[0546] Step 4:

[0547] The terminal visually displays the current location of the waste collection equipment to the user based on the displayed map data. Users can check real-time location information on the app. The input is map data, and the output is a map display on the user's terminal.

[0548] Step 5:

[0549] The server retrieves the schedules of meeting participants in a regional group using the Google Calendar API and adjusts the meeting schedule based on their availability. It analyzes the retrieved schedule data, finds matching available time slots, and sets the date. The input is the participants' calendar data, and the output is the optimal meeting date.

[0550] Step 6:

[0551] The server acquires audio data during the meeting and converts it into text in real time using the Google Cloud Speech-to-Text API. The input is audio data, and the output is text data for meeting minutes.

[0552] Step 7:

[0553] The server receives health information sent from the elderly person's device and analyzes it using a generative AI model. It uses Python and TensorFlow to determine if the values ​​are outliers and generates advice. The input is health data, and the output is a generated health advice message.

[0554] Step 8:

[0555] The server sends the generated advice to the resident's device. Firebase Cloud Messaging is used again to execute the process of notifying the resident's device of the health advice. The input is the advice message, and the output is the notification displayed on the device.

[0556] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0557] This invention combines an emotion engine with an AI system that supports community activities and personal health management. This system recognizes the user's emotions and provides more personalized feedback in community activities and health management. Specific embodiments of each function are described below.

[0558] First, the server runs an AI model responsible for emotion recognition. It analyzes user input (e.g., voice, text, and sometimes images) and extracts emotional information from it. It recognizes emotional states such as joy, sadness, and anger as emotional data. This information is used to provide feedback tailored to the current emotional state of the residents.

[0559] For example, if a user is perceived as "frustrated" when receiving a garbage collection notification, the system will simplify and make the notification more straightforward, or add words of encouragement to make it easier for the user to accept. Similarly, in a neighborhood association meeting, if a participant is feeling "anxious," the system will provide a more detailed explanation of the agenda or encourage them to ask questions in advance to ensure the meeting proceeds smoothly.

[0560] Furthermore, the emotion engine tracks users' emotional tendencies over the long term by accumulating and analyzing their emotional history in a database. This data can serve as reference information when adjusting the management policies of local communities. For example, if there are signs of a decline in overall community well-being, it can provide a starting point for suggesting events or new initiatives.

[0561] In the field of elderly care, it is possible to provide special attention to elderly individuals whose feelings of loneliness and stress are emphasized through emotion recognition. For example, if the emotion "loneliness" is detected, the server can send a message recommending communication with family and friends.

[0562] Thus, the system incorporating the emotion engine of the present invention can achieve flexible responses that take into account the user's mental state, thereby improving the quality of community activities. This will improve the quality of life for residents and contribute to the revitalization of the entire community.

[0563] The following describes the processing flow.

[0564] Step 1: The user inputs data into the device regarding their daily activities, health status, willingness to participate in community activities, etc., or speaks about these topics.

[0565] Step 2: The device sends input data or voice data to the server in real time.

[0566] Step 3: Based on the data received by the server, the emotion engine is activated and natural language processing and sentiment analysis are performed.

[0567] Step 4: The server analyzes the user's emotional state using the emotion engine, classifies the analysis results, and saves them in an emotion folder.

[0568] Step 5: The server generates the most appropriate response based on the analysis results, considering the user's state. For example, if the server detects that the user is "frustrated," it provides concise and clear information.

[0569] Step 6: The server sends the generated response to the terminal and provides personalized feedback to the user.

[0570] Step 7: The user reviews the feedback displayed on their device and enters replies or additional data into the system as needed.

[0571] Step 8: The server activates a function that stores long-term emotional data in a database and analyzes emotional trends over time.

[0572] Step 9: The server provides the aggregated emotional data to stakeholders as material for planning community activities and proposing new services.

[0573] Through this series of steps, the present invention can provide services that are sensitive to the feelings of residents and improve the level of engagement of the entire local community.

[0574] (Example 2)

[0575] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0576] In modern society, community activities and health management often fail to address the individual needs of residents, with challenges including inefficient information provision and a lack of mental care. Furthermore, there is difficulty in understanding the overall well-being and trends of the community and implementing appropriate measures based on that understanding. Traditional systems are unable to provide individualized feedback based on residents' emotional states, thus failing to contribute to the revitalization of the community and the improvement of residents' quality of life.

[0577] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0578] This invention includes a server that includes means for retrieving plans for local events from a data storage device and sending notifications to residents, means for tracking and instantly providing residents with information on the placement of garbage collection vehicles, and means for automatically resetting and notifying residents of community meeting schedules based on their availability. This improves the accuracy of information provision regarding the community surrounding residents and enables community revitalization and improvement of residents' quality of life through the use of personalized feedback.

[0579] A "data storage device" is a device used to store information such as plans for local events and to retrieve that data as needed.

[0580] "Resident" refers to an individual who lives or works within a specific area.

[0581] "Means of transmitting information" refers to means used to notify residents of information, including electronic messages and notifications.

[0582] A "waste collection vehicle" is a vehicle used to collect and transport waste within a region.

[0583] "Location information" refers to positional information that indicates where objects or people are currently located.

[0584] "Tracking" is the act of continuously monitoring the movements of a specific object or piece of information.

[0585] "Community meeting schedule" refers to the dates of meetings, events, and other gatherings held within the local community.

[0586] "Reconfiguration" refers to the act of changing existing settings and adjusting them based on new conditions or requirements.

[0587] "Textual information" refers to data that converts audio information into text, and is expressed as text.

[0588] A "document" refers to a collection of textual information compiled about a particular subject.

[0589] "Health information" refers to data related to an individual's health status, and may include body temperature, pulse rate, etc.

[0590] A "reference value" refers to a numerical value or state that is considered normal under specific conditions.

[0591] "Advice" refers to suggestions or suggestions offered in response to a specific situation or problem.

[0592] An "AI model" is a computational model that performs a specific task based on artificial intelligence.

[0593] "Personalized feedback" refers to responses and information tailored to the specific circumstances and needs of a particular user.

[0594] "Emotional history" refers to a record of past emotional states and is used to track a user's emotional tendencies.

[0595] "Means of analyzing trends" refers to methods or tools used to evaluate patterns and trends contained in collected data.

[0596] This invention incorporates an emotion engine into a system designed to support community activities and individual health management. The system aims to improve the quality of life for residents by identifying users' emotions and providing feedback based on that data.

[0597] Hardware and software to be used

[0598] This system uses two main hardware components: a server and a terminal. The server functions as a platform for running a generative AI model that performs emotion recognition. The terminal is responsible for collecting user input and sending it to the server. Data processing uses software libraries for emotion analysis and natural language processing models.

[0599] Data processing and calculations

[0600] The server inputs audio, text, and sometimes image data received from the user via the device into an emotion recognition model and analyzes the emotional information. The recognized emotional data includes a variety of states such as joy, sadness, and anger. The server then uses a generative AI model to generate personalized feedback. This feedback, optimized for the user's current emotional state, is delivered to the user via the device.

[0601] Examples of specific cases and prompt statements

[0602] For example, if the server detects that a user is experiencing stress based on their input, it can provide the user with encouraging messages or suggestions for relaxation. Furthermore, by analyzing the emotional history of the entire local community, it can suggest events to address common problems such as declining well-being.

[0603] An example of a prompt would be, "Please propose a new event to improve well-being in this area." By inputting this prompt into the AI ​​model, specific activity proposals tailored to the needs of the region can be obtained.

[0604] In this way, this system enables flexible responses that adapt to the user's emotional state, contributing to the qualitative improvement and revitalization of local communities.

[0605] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0606] Step 1:

[0607] Users input information into the system through their devices. This input data can be voice, text, or images. For example, a user might speak about their emotions or health status into their smartphone, and this information is recorded on the device.

[0608] Step 2:

[0609] The terminal sends the input data received from the user to the server. At this time, the data is formatted into a format that can be processed by the server. Specifically, voice data is converted into text data by speech recognition software.

[0610] Step 3:

[0611] The server inputs the received data into a generative AI model for emotion recognition, which then analyzes the user's emotions. Emotional information such as joy, sadness, and anger is extracted. Through this computational process, various emotional states are identified from the data.

[0612] Step 4:

[0613] The server generates personalized feedback based on emotional data. A generative AI model uses specific prompts to create appropriate messages. For example, "In this case, we recommend listening to music to relax."

[0614] Step 5:

[0615] The server generates feedback and notifies the user via their device. The user receives advice tailored to their situation and can use it to guide their actions.

[0616] Step 6:

[0617] The server stores users' emotional history in a database and analyzes long-term emotional trends. This historical data is used to extract information on overall community well-being and problems, serving as a source of information for guiding community improvement proposals.

[0618] (Application Example 2)

[0619] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0620] In community activities and health management for the elderly, there is a lack of support that takes into account the emotional state of individual users. Therefore, flexible and individualized responses are needed to improve the quality of life in the community. Furthermore, reducing feelings of loneliness and stress among the elderly is considered important, and to solve these problems, a new system is needed that recognizes emotions and enables individualized responses.

[0621] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0622] In this invention, the server includes means for obtaining a schedule of local activities from a database and sending notifications to users; means for tracking the location information of waste collection machines and providing it to users immediately; means for converting audio information during meetings into text information and generating meeting minutes; and means for recognizing the emotional state of users and providing personalized support in accordance with those emotions. This enables individualized responses in accordance with users' emotions, as well as the smooth implementation of local activities and improved health management for the elderly.

[0623] A "database" is an information processing system that stores various information such as schedules for community activities and information about users, and allows for searching and referencing as needed.

[0624] "Community activities" refer to various events, gatherings, volunteer activities, and community services within the local community, and include organized actions to promote resident participation and cooperation.

[0625] A "waste collection machine" refers to a machine used to collect waste from a community and transport it to a designated processing facility. Specifically, this includes garbage trucks.

[0626] "Location information" refers to data that indicates the geographical location of a specific object or person, and is usually expressed as coordinates or an address.

[0627] "Auditory information" refers to information that can be acquired through hearing, such as human speech and ambient sounds. It is mainly converted into digital data using recording technology.

[0628] "Textual information" refers to linguistic information digitized using character codes, and takes a form that is easy to communicate and store.

[0629] "Meeting minutes" are documents that record the content of discussions and decisions made at meetings or gatherings, and organize them in a way that allows for later reference.

[0630] "Health information" refers to data about an individual's health status, including physiological data such as vital signs and diagnostic results.

[0631] "Emotional state" refers to an individual's psychological state as a response to a specific situation or stimulus, and includes states such as joy, sadness, anger, and anxiety.

[0632] "Individualized support" refers to special responses or services provided in a way that is most appropriate, taking into account the needs and emotional state of each user.

[0633] In this invention, a server, which is an information processing device, combines various hardware and software to provide services that support community activities and health management. The server uses a database system to obtain the schedule of community activities and sends notifications to the user's terminal. It also uses GPS tracking technology to immediately obtain the location information of waste collection machines and provides it to the user.

[0634] The server uses speech processing software to convert audio information during meetings into text and automatically generates meeting minutes. This text conversion is achieved by utilizing a speech recognition engine. Furthermore, the server is equipped with an emotion recognition system that processes audio and image data acquired through the user's terminal to determine the user's emotional state. This function enables personalized support tailored to emotions, and in particular, it can improve the quality of life for the elderly by providing appropriate advice based on health information.

[0635] This technology can be useful, for example, in suggesting relaxing music and activities to alleviate feelings of loneliness that older adults often experience. Specifically, if the system detects that a user is feeling lonely, it can send notifications suggesting communication with family and friends, or relaxation activities such as sunbathing or taking a walk.

[0636] Furthermore, an example of generating prompt text in these processes using a generative AI model is: "I want to develop a care support application to alleviate feelings of loneliness for the elderly. Please give me advice on writing a program that analyzes the user's emotions in real time and generates suggestions to encourage communication with family members when necessary."

[0637] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0638] Step 1:

[0639] The server retrieves the schedule of local activities from the database and sends a notification to the user's device. It sends a query about local activities to the database as input, and the activity schedule information is obtained as output. Based on this information, a notification is generated in text format and sent to the user's device.

[0640] Step 2:

[0641] The server uses GPS tracking technology to collect location information from waste collection machines and provides this location information to the user's device. It receives real-time location data from waste collection machines as input and outputs location information that can be displayed on a map. This location information is displayed as a graphical interface on the user's device.

[0642] Step 3:

[0643] The server uses audio processing software to acquire audio information during a meeting and converts it into text. It receives audio data from a microphone as input and outputs a text-based meeting transcript. A speech recognition engine analyzes the sound wave data and converts it into text.

[0644] Step 4:

[0645] The server acquires voice and image data from the user and determines the user's emotional state through an emotion recognition system. It receives the user's voice and facial image data as input, and evaluates their emotional state as output. An emotion analysis algorithm processes the data and assigns emotion labels such as joy or sadness.

[0646] Step 5:

[0647] Based on the user's emotional state, the server provides personalized support. It receives the results of an emotional analysis as input and generates optimal suggestions and notifications for the user as output. For example, if the server determines the user is feeling lonely, a message encouraging communication is sent to the device, supporting the user's actions.

[0648] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0649] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0650] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0651] [Fourth Embodiment]

[0652] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0653] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0654] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0655] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0656] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0657] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0658] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0659] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0660] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0661] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0662] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0663] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0664] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0665] This invention provides an AI agent system that supports community activities. This system primarily functions in three areas: garbage management, neighborhood association operation support, and elderly care support.

[0666] First, the server extracts garbage collection schedule information from a local activity database and sends a notification to residents' devices based on this information. Residents' devices receive this notification and display a reminder at the set time. This entire process helps residents put out their garbage on the correct dates and times. For example, if combustible waste is collected on Mondays and Thursdays, the server automatically notifies residents, and their devices send a reminder in the morning.

[0667] Next, the server receives location data from GPS devices installed on garbage trucks and makes it accessible to residents as map information. Residents can then view this information through an application and monitor in real time whether their garbage is being collected properly. For example, when a resident opens the app, they can see that a garbage truck is heading towards their area.

[0668] The neighborhood association's operational support system incorporates a server that checks each participant's availability, automatically adjusts meeting schedules, and sends notifications. Furthermore, during meetings, audio data is automatically transcribed, generating meeting minutes. These minutes are shared with participants, who can review them on their devices and make corrections as needed.

[0669] Finally, as part of support for the elderly, a system has been introduced that allows users to input daily health data using a terminal and send it to a server. The server analyzes this data, compares it to standard values, evaluates the user's health status, and generates advice. For example, if body temperature or blood pressure exceeds a certain range, a health alert is automatically sent.

[0670] This system is designed to streamline local community activities and reduce the burden on residents and stakeholders. In this way, it can help address depopulation in the region and strengthen cooperation among residents.

[0671] The following describes the processing flow.

[0672] Step 1: The server accesses the local activity schedule database to retrieve the latest garbage collection schedule information.

[0673] Step 2: The server uses the residents' contact information to send the retrieved garbage collection schedule as a push notification to each resident's device.

[0674] Step 3: Set up the device to display received notifications and provide reminders to residents at the specified time.

[0675] Step 4: The user checks the notification on their device and prepares to take out the trash if necessary.

[0676] Step 5: The server collects location information in real time from GPS devices installed in the garbage trucks.

[0677] Step 6: The server analyzes the location information it has collected and formats the data to display the route of the collection vehicle on a map.

[0678] Step 7: Users check the current location and planned route of the garbage truck via a smartphone app.

[0679] Step 8: For neighborhood association operation support, the server retrieves participants' calendar information and automatically adjusts the optimal meeting schedule.

[0680] Step 9: The server notifies participants of the meeting date as a calendar entry on their devices.

[0681] Step 10: The user checks the meeting schedule on their device and makes manual changes if necessary.

[0682] Step 11: During the meeting, the user collects audio data using a recording device.

[0683] Step 12: The server receives the recorded audio data and converts it into text data using speech recognition technology.

[0684] Step 13: The server checks the converted text data and automatically generates the first version of the meeting minutes.

[0685] Step 14: The user reviews the initial version of the meeting minutes, makes any necessary revisions, and finalizes the official version.

[0686] Step 15: Every morning, the elderly person uses a device to input health data (e.g., body temperature, blood pressure) and sends it to the server.

[0687] Step 16: The server analyzes the received health data and evaluates the health status by comparing it to standard health values.

[0688] Step 17: Based on the evaluation results, the server generates necessary advice and notifies the user's terminal.

[0689] Step 18: Review the advice the user received and use it to help manage their health.

[0690] (Example 1)

[0691] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0692] In local communities, there is a need for systems that efficiently support diverse activities in which residents participate and the health management of the elderly. However, existing methods suffer from delays in information dissemination, difficulties in coordination, and limitations in providing individualized support. Therefore, efficiently managing information related to community activities and providing appropriate support to each resident is a challenge.

[0693] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0694] In this invention, the server includes means for acquiring scheduled information on local activities from an information storage means via a communication medium and transmitting notifications to a receiving device; means for collecting location information of mobile objects and providing it to residents immediately; and means for automatically adjusting and notifying participants of the meeting date and time of local organizations based on their available time. This enables the rapid provision of information on local activities, the smooth coordination of various events, and the efficient provision of individual support.

[0695] "Communication medium" refers to the means and routes for sending and receiving information, and is a general term that includes wired and wireless technologies.

[0696] "Information storage means" refers to devices and systems used to store and manage data, such as databases.

[0697] A "receiving device" refers to a device used to receive information transmitted from a server, and includes smartphones and tablets.

[0698] "Location information of a moving object" refers to data indicating the geographical location of a specific object or vehicle, and is obtained using technologies such as GPS.

[0699] A "community group" refers to a collection or organization of residents in a specific geographical area, such as a neighborhood association or a local residents' association.

[0700] "Meeting date and time" is a term that refers to the date and time when a particular activity or event takes place.

[0701] "Available time" refers to information indicating the dates and times when an individual or group can participate in a particular activity.

[0702] "Means of notification" refers to the methods and technologies used to transmit information to recipients, such as email and push notifications.

[0703] This invention is a system for efficiently supporting community activities, and mainly consists of a server, terminals, and users. The server accesses a region-specific information storage means and obtains scheduled information on community activities. This information is mainly managed using a database management system. The server transmits the obtained information as a notification to a receiving device via a communication medium. The notification is provided via an internet-based protocol (e.g., HTTP) and communication service. As a specific example, it is conceivable that a reminder of neighborhood association events would be sent to residents' terminals every Monday.

[0704] Furthermore, the server collects location information of moving objects and provides this information to residents in real time. This involves data processing using GPS devices and map APIs. The terminal can then visually present this information to residents through an application. For example, when a resident opens the app, the current location of the garbage truck is displayed on a map.

[0705] Furthermore, to support the efficient management of local group meetings, the server collects each participant's availability and automatically adjusts the optimal meeting time. This utilizes a calendar API to aggregate individual schedule information and perform calculations to derive the optimal solution. During the meeting, audio information is acquired using a recording device and converted into text using speech recognition technology. The generated text information, which serves as meeting minutes, is shared with users' devices for later review and correction.

[0706] Finally, health information is entered by the elderly person via their device and sent to the server. Within the server, a data analysis program compares the health information to standard values ​​and assesses their health status. The assessment results and any necessary advice are then notified to the user's device. This could potentially result in an alert being sent to the elderly person, such as, "Your body temperature is high today, please rest."

[0707] An example of a prompt message is, "Please briefly explain the process for automatically scheduling neighborhood association meetings." This invention provides an effective means to comprehensively support the lives of residents and facilitate community activities.

[0708] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0709] Step 1:

[0710] The server accesses a database of local activities and extracts schedule information relevant to residents. It uses tables and query conditions within the database as input and retrieves the schedule information in JSON format as output. Based on this information, the server prepares notifications and organizes the data for sending them via communication media. Specifically, it extracts information such as "The next neighborhood association meeting is next Wednesday" from the database.

[0711] Step 2:

[0712] The device receives notifications sent from the server and displays reminders at the appropriate time based on the user's schedule. It receives notification data from the server as input and displays reminders on the user's screen as output. Specifically, a pop-up message appears on the device saying, "There is a neighborhood association meeting tomorrow."

[0713] Step 3:

[0714] The server receives current location information from the garbage truck's GPS device. It takes location data from the GPS as input and updates that data as map information in real time as output. The server uses a map API to generate visual location information and provides it to the terminal. Specifically, it displays "Garbage truck passing through area A" on the map.

[0715] Step 4:

[0716] The device displays map information provided by the server, allowing the user to check the location of the garbage truck in real time. It receives map data from the server as input and displays routes and location icons on the map application as output. In practice, the app displays updated information such as "10 minutes until the garbage truck arrives."

[0717] Step 5:

[0718] The server references the calendars of meeting participants and calculates the optimal meeting date. It takes participants' calendar information (a list of their free time slots) as input and generates a new, adjusted meeting date as output. Specifically, it calculates information such as, "The next meeting is scheduled for Friday at 3 PM."

[0719] Step 6:

[0720] The device receives updated meeting schedules from the server and notifies participants. It takes the new meeting schedule as input and adds it as a calendar event as output. Specifically, a notification appears in the device's calendar app stating, "A new meeting has been added."

[0721] Step 7:

[0722] The user uses their device to input their health information and sends it to the server. Inputs include health indicators such as body temperature and blood pressure, and output is health data sent to the server. For example, the user might enter "36.7 degrees" into the app and tap the send button.

[0723] Step 8:

[0724] The server analyzes the received health data and evaluates the user's health status by comparing it to standard values. It uses health data submitted by the user as input and generates analysis results and advice as output. For example, it might create a message such as, "Your blood pressure is a little high, so please drink more water."

[0725] Step 9:

[0726] The device receives health analysis results from the server and displays an alert to the user. It receives advice messages from the server as input and displays a warning directly on the user's screen as output. Specifically, the app's notification area displays "Health Warning: Your body temperature is high today, please be careful."

[0727] (Application Example 1)

[0728] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0729] To efficiently manage and promote community activities, the rapid provision of various information and smooth communication are essential. However, conventional methods have problems such as missing garbage collection times, difficulty in coordinating meeting schedules, and inadequate health management for the elderly. Furthermore, tracking residents' health status and immediate response to emergencies are insufficient, necessitating a comprehensive system to improve community safety and quality of life.

[0730] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0731] In this invention, the server includes means for retrieving the schedule of community activities from a storage device and sending notifications to residents; means for tracking the location information of waste collection devices and providing it to residents immediately; means for automatically adjusting and notifying community group meeting schedules based on the availability of participants; and means for providing health monitoring functions to residents' terminals and notifying them of medical-related information. This makes community activities more efficient, reduces the burden on residents, and enables a rapid response in emergencies.

[0732] "Community activities" refer to various events and activities related to daily life that are carried out jointly by residents and organizations within a local community.

[0733] A "storage device" is a device that collects and stores information as a database and retrieves and uses it as needed.

[0734] "Residents" refers to people who live in a specific area or building, and in this invention, they play the role of information recipients.

[0735] "Waste collection equipment" refers to vehicles or devices used to collect specific types of waste and process them appropriately.

[0736] "Location information" refers to the current geographical location of a specific object or person, and is usually expressed as latitude and longitude coordinates.

[0737] A "local group" refers to a collection of people or organizations that share social relationships within a specific region.

[0738] "Free time" refers to periods in one's schedule when there are no other scheduled events.

[0739] "Health information" refers to data that indicates an individual's physical condition, including body temperature, blood pressure, and pulse rate.

[0740] "Medical information" refers to advice or warnings regarding an individual's health condition or illness, or important notices from healthcare providers.

[0741] "Terminal" generally refers to electronic devices such as smartphones, personal computers, and tablets, and in this invention, it refers to a device that displays information or inputs data.

[0742] The server retrieves the schedule of local activities from a storage device and runs a program that sends notifications to residents' terminals. This program uses MySQL as its database management system to organize and manage schedule information that differs for each region. Firebase Cloud Messaging is used for communication, ensuring that notifications are sent efficiently.

[0743] Furthermore, the server obtains location information from GPS devices installed on the waste collection equipment and immediately provides that information to residents' devices using the Google Maps API. This allows residents to monitor in real time how close the waste collection equipment is to their area.

[0744] For scheduling meetings within local groups, the server automatically checks participants' availability and adjusts meeting dates. Schedule information is managed using the Google Calendar API and is designed to ensure timely notifications. Furthermore, the Google Cloud Speech-to-Text API is used for real-time transcription of meeting speeches. This automatically generates meeting minutes, which are then shared with participants.

[0745] In managing health information, residents use their smartphones to input daily health data and send it to a server. This information is analyzed using Python and TensorFlow, and an AI model provides residents with advice compared to standard values. Based on this, residents can understand their own health status and consult with medical institutions as needed.

[0746] As a concrete example, suppose an elderly person uses their smartphone every morning to record their body temperature and blood pressure and sends it to a server. Once the data reaches the server, a Python script is executed, an AI model evaluates any anomalies, and automatically sends advice such as, "Your morning body temperature is higher than normal. Please consider consulting a doctor," if necessary.

[0747] An example of a prompt might be: "Explain how residents can use the health monitoring features provided by the Smart City Assistant to understand their health status in real time and receive necessary medical advice."

[0748] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0749] Step 1:

[0750] The server retrieves schedule information for local activities from a storage device. It queries the schedule information stored in the MySQL database and extracts the notification target data for the specified time. The input is the local ID, and the output is the corresponding schedule data.

[0751] Step 2:

[0752] The server uses Firebase Cloud Messaging to retrieve schedule information and send it to the residents' devices. The extracted schedule data is formatted as a message and sent as a notification to each resident's registered device. The input is the schedule data, and the output is the notification displayed on the resident's device.

[0753] Step 3:

[0754] The server obtains location information for waste collection equipment. It receives current coordinate data from a GPS device installed on the collection equipment and creates map data for visualization using the Google Maps API. The input is GPS coordinates, and the output is map data.

[0755] Step 4:

[0756] The terminal visually displays the current location of the waste collection equipment to the user based on the displayed map data. Users can check real-time location information on the app. The input is map data, and the output is a map display on the user's terminal.

[0757] Step 5:

[0758] The server retrieves the schedules of meeting participants in a regional group using the Google Calendar API and adjusts the meeting schedule based on their availability. It analyzes the retrieved schedule data, finds matching available time slots, and sets the date. The input is the participants' calendar data, and the output is the optimal meeting date.

[0759] Step 6:

[0760] The server acquires audio data during the meeting and converts it into text in real time using the Google Cloud Speech-to-Text API. The input is audio data, and the output is text data for meeting minutes.

[0761] Step 7:

[0762] The server receives health information sent from the elderly person's device and analyzes it using a generative AI model. It uses Python and TensorFlow to determine if the values ​​are outliers and generates advice. The input is health data, and the output is a generated health advice message.

[0763] Step 8:

[0764] The server sends the generated advice to the resident's device. Firebase Cloud Messaging is used again to execute the process of notifying the resident's device of the health advice. The input is the advice message, and the output is the notification displayed on the device.

[0765] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0766] This invention combines an emotion engine with an AI system that supports community activities and personal health management. This system recognizes the user's emotions and provides more personalized feedback in community activities and health management. Specific embodiments of each function are described below.

[0767] First, the server runs an AI model responsible for emotion recognition. It analyzes user input (e.g., voice, text, and sometimes images) and extracts emotional information from it. It recognizes emotional states such as joy, sadness, and anger as emotional data. This information is used to provide feedback tailored to the current emotional state of the residents.

[0768] For example, if a user is perceived as "frustrated" when receiving a garbage collection notification, the system will simplify and make the notification more straightforward, or add words of encouragement to make it easier for the user to accept. Similarly, in a neighborhood association meeting, if a participant is feeling "anxious," the system will provide a more detailed explanation of the agenda or encourage them to ask questions in advance to ensure the meeting proceeds smoothly.

[0769] Furthermore, the emotion engine tracks users' emotional tendencies over the long term by accumulating and analyzing their emotional history in a database. This data can serve as reference information when adjusting the management policies of local communities. For example, if there are signs of a decline in overall community well-being, it can provide a starting point for suggesting events or new initiatives.

[0770] In the field of elderly care, it is possible to provide special attention to elderly individuals whose feelings of loneliness and stress are emphasized through emotion recognition. For example, if the emotion "loneliness" is detected, the server can send a message recommending communication with family and friends.

[0771] Thus, the system incorporating the emotion engine of the present invention can achieve flexible responses that take into account the user's mental state, thereby improving the quality of community activities. This will improve the quality of life for residents and contribute to the revitalization of the entire community.

[0772] The following describes the processing flow.

[0773] Step 1: The user inputs data into the device regarding their daily activities, health status, willingness to participate in community activities, etc., or speaks about these topics.

[0774] Step 2: The device sends input data or voice data to the server in real time.

[0775] Step 3: Based on the data received by the server, the emotion engine is activated and natural language processing and sentiment analysis are performed.

[0776] Step 4: The server analyzes the user's emotional state using the emotion engine, classifies the analysis results, and saves them in an emotion folder.

[0777] Step 5: The server generates the most appropriate response based on the analysis results, considering the user's state. For example, if the server detects that the user is "frustrated," it provides concise and clear information.

[0778] Step 6: The server sends the generated response to the terminal and provides personalized feedback to the user.

[0779] Step 7: The user reviews the feedback displayed on their device and enters replies or additional data into the system as needed.

[0780] Step 8: The server activates a function that stores long-term emotional data in a database and analyzes emotional trends over time.

[0781] Step 9: The server provides the aggregated emotional data to stakeholders as material for planning community activities and proposing new services.

[0782] Through this series of steps, the present invention can provide services that are sensitive to the feelings of residents and improve the level of engagement of the entire local community.

[0783] (Example 2)

[0784] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0785] In modern society, community activities and health management often fail to address the individual needs of residents, with challenges including inefficient information provision and a lack of mental care. Furthermore, there is difficulty in understanding the overall well-being and trends of the community and implementing appropriate measures based on that understanding. Traditional systems are unable to provide individualized feedback based on residents' emotional states, thus failing to contribute to the revitalization of the community and the improvement of residents' quality of life.

[0786] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0787] This invention includes a server that includes means for retrieving plans for local events from a data storage device and sending notifications to residents, means for tracking and instantly providing residents with information on the placement of garbage collection vehicles, and means for automatically resetting and notifying residents of community meeting schedules based on their availability. This improves the accuracy of information provision regarding the community surrounding residents and enables community revitalization and improvement of residents' quality of life through the use of personalized feedback.

[0788] A "data storage device" is a device used to store information such as plans for local events and to retrieve that data as needed.

[0789] "Resident" refers to an individual who lives or works within a specific area.

[0790] "Means of transmitting information" refers to means used to notify residents of information, including electronic messages and notifications.

[0791] A "waste collection vehicle" is a vehicle used to collect and transport waste within a region.

[0792] "Location information" refers to positional information that indicates where objects or people are currently located.

[0793] "Tracking" is the act of continuously monitoring the movements of a specific object or piece of information.

[0794] "Community meeting schedule" refers to the dates of meetings, events, and other gatherings held within the local community.

[0795] "Reconfiguration" refers to the act of changing existing settings and adjusting them based on new conditions or requirements.

[0796] "Textual information" refers to data that converts audio information into text, and is expressed as text.

[0797] A "document" refers to a collection of textual information compiled about a particular subject.

[0798] "Health information" refers to data related to an individual's health status, and may include body temperature, pulse rate, etc.

[0799] A "reference value" refers to a numerical value or state that is considered normal under specific conditions.

[0800] "Advice" refers to suggestions or suggestions offered in response to a specific situation or problem.

[0801] An "AI model" is a computational model that performs a specific task based on artificial intelligence.

[0802] "Personalized feedback" refers to responses and information tailored to the specific circumstances and needs of a particular user.

[0803] "Emotional history" refers to a record of past emotional states and is used to track a user's emotional tendencies.

[0804] "Means of analyzing trends" refers to methods or tools used to evaluate patterns and trends contained in collected data.

[0805] This invention incorporates an emotion engine into a system designed to support community activities and individual health management. The system aims to improve the quality of life for residents by identifying users' emotions and providing feedback based on that data.

[0806] Hardware and software to be used

[0807] This system uses two main hardware components: a server and a terminal. The server functions as a platform for running a generative AI model that performs emotion recognition. The terminal is responsible for collecting user input and sending it to the server. Data processing uses software libraries for emotion analysis and natural language processing models.

[0808] Data processing and calculations

[0809] The server inputs audio, text, and sometimes image data received from the user via the device into an emotion recognition model and analyzes the emotional information. The recognized emotional data includes a variety of states such as joy, sadness, and anger. The server then uses a generative AI model to generate personalized feedback. This feedback, optimized for the user's current emotional state, is delivered to the user via the device.

[0810] Examples of specific cases and prompt statements

[0811] For example, if the server detects that a user is experiencing stress based on their input, it can provide the user with encouraging messages or suggestions for relaxation. Furthermore, by analyzing the emotional history of the entire local community, it can suggest events to address common problems such as declining well-being.

[0812] An example of a prompt would be, "Please propose a new event to improve well-being in this area." By inputting this prompt into the AI ​​model, specific activity proposals tailored to the needs of the region can be obtained.

[0813] In this way, this system enables flexible responses that adapt to the user's emotional state, contributing to the qualitative improvement and revitalization of local communities.

[0814] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0815] Step 1:

[0816] Users input information into the system through their devices. This input data can be voice, text, or images. For example, a user might speak about their emotions or health status into their smartphone, and this information is recorded on the device.

[0817] Step 2:

[0818] The terminal sends the input data received from the user to the server. At this time, the data is formatted into a format that can be processed by the server. Specifically, voice data is converted into text data by speech recognition software.

[0819] Step 3:

[0820] The server inputs the received data into a generative AI model for emotion recognition, which then analyzes the user's emotions. Emotional information such as joy, sadness, and anger is extracted. Through this computational process, various emotional states are identified from the data.

[0821] Step 4:

[0822] The server generates personalized feedback based on emotional data. A generative AI model uses specific prompts to create appropriate messages. For example, "In this case, we recommend listening to music to relax."

[0823] Step 5:

[0824] The server generates feedback and notifies the user via their device. The user receives advice tailored to their situation and can use it to guide their actions.

[0825] Step 6:

[0826] The server stores users' emotional history in a database and analyzes long-term emotional trends. This historical data is used to extract information on overall community well-being and problems, serving as a source of information for guiding community improvement proposals.

[0827] (Application Example 2)

[0828] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0829] In community activities and health management for the elderly, there is a lack of support that takes into account the emotional state of individual users. Therefore, flexible and individualized responses are needed to improve the quality of life in the community. Furthermore, reducing feelings of loneliness and stress among the elderly is considered important, and to solve these problems, a new system is needed that recognizes emotions and enables individualized responses.

[0830] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0831] In this invention, the server includes means for obtaining a schedule of local activities from a database and sending notifications to users; means for tracking the location information of waste collection machines and providing it to users immediately; means for converting audio information during meetings into text information and generating meeting minutes; and means for recognizing the emotional state of users and providing personalized support in accordance with those emotions. This enables individualized responses in accordance with users' emotions, as well as the smooth implementation of local activities and improved health management for the elderly.

[0832] A "database" is an information processing system that stores various information such as schedules for community activities and information about users, and allows for searching and referencing as needed.

[0833] "Community activities" refer to various events, gatherings, volunteer activities, and community services within the local community, and include organized actions to promote resident participation and cooperation.

[0834] A "waste collection machine" refers to a machine used to collect waste from a community and transport it to a designated processing facility. Specifically, this includes garbage trucks.

[0835] "Location information" refers to data that indicates the geographical location of a specific object or person, and is usually expressed as coordinates or an address.

[0836] "Auditory information" refers to information that can be acquired through hearing, such as human speech and ambient sounds. It is mainly converted into digital data using recording technology.

[0837] "Textual information" refers to linguistic information digitized using character codes, and takes a form that is easy to communicate and store.

[0838] "Meeting minutes" are documents that record the content of discussions and decisions made at meetings or gatherings, and organize them in a way that allows for later reference.

[0839] "Health information" refers to data about an individual's health status, including physiological data such as vital signs and diagnostic results.

[0840] "Emotional state" refers to an individual's psychological state as a response to a specific situation or stimulus, and includes states such as joy, sadness, anger, and anxiety.

[0841] "Individualized support" refers to special responses or services provided in a way that is most appropriate, taking into account the needs and emotional state of each user.

[0842] In this invention, a server, which is an information processing device, combines various hardware and software to provide services that support community activities and health management. The server uses a database system to obtain the schedule of community activities and sends notifications to the user's terminal. It also uses GPS tracking technology to immediately obtain the location information of waste collection machines and provides it to the user.

[0843] The server uses speech processing software to convert audio information during meetings into text and automatically generates meeting minutes. This text conversion is achieved by utilizing a speech recognition engine. Furthermore, the server is equipped with an emotion recognition system that processes audio and image data acquired through the user's terminal to determine the user's emotional state. This function enables personalized support tailored to emotions, and in particular, it can improve the quality of life for the elderly by providing appropriate advice based on health information.

[0844] This technology can be useful, for example, in suggesting relaxing music and activities to alleviate feelings of loneliness that older adults often experience. Specifically, if the system detects that a user is feeling lonely, it can send notifications suggesting communication with family and friends, or relaxation activities such as sunbathing or taking a walk.

[0845] Furthermore, an example of generating prompt text in these processes using a generative AI model is: "I want to develop a care support application to alleviate feelings of loneliness for the elderly. Please give me advice on writing a program that analyzes the user's emotions in real time and generates suggestions to encourage communication with family members when necessary."

[0846] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0847] Step 1:

[0848] The server retrieves the schedule of local activities from the database and sends a notification to the user's device. It sends a query about local activities to the database as input, and the activity schedule information is obtained as output. Based on this information, a notification is generated in text format and sent to the user's device.

[0849] Step 2:

[0850] The server uses GPS tracking technology to collect location information from waste collection machines and provides this location information to the user's device. It receives real-time location data from waste collection machines as input and outputs location information that can be displayed on a map. This location information is displayed as a graphical interface on the user's device.

[0851] Step 3:

[0852] The server uses audio processing software to acquire audio information during a meeting and converts it into text. It receives audio data from a microphone as input and outputs a text-based meeting transcript. A speech recognition engine analyzes the sound wave data and converts it into text.

[0853] Step 4:

[0854] The server acquires voice and image data from the user and determines the user's emotional state through an emotion recognition system. It receives the user's voice and facial image data as input, and evaluates their emotional state as output. An emotion analysis algorithm processes the data and assigns emotion labels such as joy or sadness.

[0855] Step 5:

[0856] Based on the user's emotional state, the server provides personalized support. It receives the results of an emotional analysis as input and generates optimal suggestions and notifications for the user as output. For example, if the server determines the user is feeling lonely, a message encouraging communication is sent to the device, supporting the user's actions.

[0857] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0858] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0859] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0860] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0861] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0862] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0863] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0864] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0865] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0866] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0867] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0868] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0869] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0870] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0871] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0872] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0873] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0874] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0875] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0876] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0877] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0878] The following is further disclosed regarding the embodiments described above.

[0879] (Claim 1)

[0880] A means of retrieving the schedule of local activities from a database and sending notifications to residents,

[0881] A means of tracking the location of garbage trucks and providing this information to residents in real time,

[0882] A means of automatically adjusting and notifying participants of local community meeting schedules based on their schedules,

[0883] A method for converting audio data from a meeting into text data and generating meeting minutes,

[0884] A means of receiving health data from the elderly and providing advice by comparing it to standard values,

[0885] A system that includes this.

[0886] (Claim 2)

[0887] The system according to claim 1, further comprising means for prompting residents to input their health status and notifying residents of advice via their devices.

[0888] (Claim 3)

[0889] The system according to claim 1, further comprising means for monitoring the movements of residents and notifying emergency contacts if an anomaly is detected.

[0890] "Example 1"

[0891] (Claim 1)

[0892] A means for obtaining information on the schedule of local activities from an information storage means via a communication medium and transmitting a notification to a receiving device,

[0893] A means of collecting location information of moving objects and providing it to residents immediately,

[0894] A means of automatically adjusting and notifying participants of the meeting time and date for local groups based on their available time,

[0895] A means for converting audio information in a group into text information and generating a record,

[0896] A means of receiving health information from the elderly and providing advice by comparing it with reference values,

[0897] A system that includes this.

[0898] (Claim 2)

[0899] The system according to claim 1, further comprising means for prompting residents to report their health status and notifying residents of advice via their receiving devices.

[0900] (Claim 3)

[0901] The system according to claim 1, further comprising means for monitoring the movements of residents and notifying emergency contacts if an abnormality is detected.

[0902] "Application Example 1"

[0903] (Claim 1)

[0904] A means of retrieving the schedule of community activities from a storage device and sending notifications to residents,

[0905] A means of tracking the location information of waste collection equipment and providing it to residents immediately,

[0906] A means of automatically adjusting and notifying participants of meeting schedules for local groups based on their availability,

[0907] A means of converting audio information from a meeting into text information and generating meeting minutes,

[0908] A means of receiving health information from the elderly and providing advice by comparing it with standard values,

[0909] A means of providing health monitoring functions to residents' devices and notifying them of medical-related information,

[0910] A system that includes this.

[0911] (Claim 2)

[0912] The system according to claim 1, further comprising means for prompting residents to input health status data and notifying residents of advice via their devices.

[0913] (Claim 3)

[0914] The system according to claim 1, further comprising means for monitoring the movements of residents and notifying emergency contacts if an abnormality is detected.

[0915] "Example 2 of combining an emotion engine"

[0916] (Claim 1)

[0917] A means of retrieving plans for local events from a data storage device and sending notifications to residents,

[0918] A means to track the location of waste collection vehicles and provide this information to residents instantly,

[0919] A means of automatically rescheduling and notifying participants of community gatherings based on their availability,

[0920] A means of converting audio information during a meeting into text information and creating a document,

[0921] A means of receiving health information from the elderly, comparing it to reference values, and providing advice,

[0922] A means of having an AI model analyze the user's emotional state, generate personalized feedback, and notify the resident,

[0923] A means of accumulating emotional history and analyzing trends across the entire region,

[0924] A system that includes this.

[0925] (Claim 2)

[0926] The system according to claim 1, further comprising means for prompting residents to input their physical condition and notifying the residents of personalized advice on their display devices.

[0927] (Claim 3)

[0928] The system according to claim 1, further comprising means for monitoring the activities of residents and notifying emergency contacts if an anomaly is detected.

[0929] "Application example 2 when combining with an emotional engine"

[0930] (Claim 1)

[0931] A means of retrieving the schedule of local activities from a database and sending notifications to users,

[0932] A means of tracking the location information of waste collection machines and providing it to users immediately,

[0933] A means of automatically adjusting and notifying participants of local group meeting schedules based on their schedules,

[0934] A means of converting audio information during a meeting into text information and generating meeting minutes,

[0935] A means of obtaining health information from the elderly and providing advice by comparing it with reference values,

[0936] A means of recognizing the emotional state of the user and providing individualized support in accordance with that emotion,

[0937] A system that includes this.

[0938] (Claim 2)

[0939] The system according to claim 1, further comprising means for prompting the user to input their health status and notifying the user of advice via their communication terminal.

[0940] (Claim 3)

[0941] The system according to claim 1, further comprising means for monitoring user activity and notifying emergency contacts if an anomaly is detected. [Explanation of Symbols]

[0942] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of retrieving the schedule of community activities from a storage device and sending notifications to residents, A means of tracking the location information of waste collection equipment and providing it to residents immediately, A means of automatically adjusting and notifying participants of meeting schedules for local groups based on their availability, A means of converting audio information from a meeting into text information and generating meeting minutes, A means of receiving health information from the elderly and providing advice by comparing it with standard values, A means of providing health monitoring functions to residents' devices and notifying them of medical-related information, A system that includes this.

2. The system according to claim 1, further comprising means for prompting residents to input health status data and notifying residents of advice via their terminals.

3. The system according to claim 1, further comprising means for monitoring the movements of residents and notifying emergency contacts if an abnormality is detected.

Images