system

Abstract

Provide a system. Solution: A method for receiving information entered by the field activity team and having AI analyze and organize that information; Next, we need a way to store the organized information in a database and display it on a dashboard. A means for the server to periodically aggregate all information and visually display the overall information; means for sending notifications to a plurality of user devices; A means for AI to automatically respond to inquiries from field teams, A system including means for making the global information accessible to a user device.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] This system solves the problem of complicated information sharing between the control team and the field activity team when a disaster occurs. Specifically, it takes time for the control team to interview and organize information from the field activity team, and it is difficult for the field activity team to grasp the overall situation, resulting in the same inquiries being repeated multiple times. These problems can reduce the efficiency of recovery activities. [Means for solving the problem]

[0005] This invention provides a system that receives information entered by the field operation team, analyzes and organizes that information using AI, stores the organized information in a database and displays it on a dashboard, periodically aggregates all information using a server and visually displays the overall information, sends notifications to multiple user devices, automatically responds to inquiries from the field operation team using AI, and makes the overall information accessible from user devices. This system facilitates information sharing between the control team and the field operation team, improving the efficiency of disaster response.

[0006] The "on-site activity team" is a team that carries out rescue operations and reports on the damage situation at the scene of a disaster.

[0007] The "control team" is the team that takes command of disaster response, organizes information, and makes decisions.

[0008] "Information collaboration" is the process of sharing necessary information between different teams and devices and communicating effectively.

[0009] "AI" stands for artificial intelligence, which has the ability to analyze data, automatically organize information, and answer inquiries.

[0010] A database is a system that organizes and stores information and allows it to be quickly retrieved when needed.

[0011] A "dashboard" is an interface that visualizes and displays multiple pieces of information in a unified manner.

[0012] "User devices" are terminals such as computers and smartphones used by the control team and field teams.

[0013] A "notification" is an alert or message sent to a user device when new information is generated.

[0014] "Organization" is the process of systematically structuring received information based on certain rules and categories.

[0015] "Aggregation" refers to combining multiple different pieces of information into one large piece of information.

[0016] An "inquiry" is when a field team asks the AI ​​a question seeking specific information.

[0017] "Analysis" is the process of dissecting information to transform raw data into an understandable form.

[0018] "Accessible" refers to a state in which users can easily reach the information they need at any time. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention is a system for facilitating information sharing between the control team and the field activity team. The program processing in this system will be explained below.

[0041] The system consists of devices used by the control team and field teams, and a central server that manages information. The server uses AI to receive, analyze, organize, and share appropriate information. The system also sends real-time notifications to user devices, ensuring that users always have access to the latest information.

[0042] Program processing flow (natural language explanation)

[0043] 1. User Authentication

[0044] Device: The user launches the app and enters their ID and password on the login screen.

[0045] Terminal: Sends the entered ID and password to the server.

[0046] Server: Checks the received user information in the database and returns the authentication result to the terminal.

[0047] Terminal: If authentication is successful, grant the user access to the dashboard.

[0048] 2. Information gathering and interviews

[0049] User (on-site action team): Reports the situation on-site in chat format (e.g., "The building has collapsed. 10 people injured.").

[0050] Terminal: Sends user input to the server.

[0051] Server: Receives information, and the AI ​​analyzes and organizes it into detailed items (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10.").

[0052] Server: Stores organized information in a database and displays it on a dashboard.

[0053] Terminal (control team): Organized information is displayed in real time on the dashboard.

[0054] 3. Automated response to inquiries

[0055] User (on-site response team): Asks the AI ​​for the necessary information (e.g., "Where is the nearest evacuation shelter?").

[0056] Terminal: Sends the query to the server.

[0057] Server: The AI ​​compares the database with the latest information and generates an appropriate answer (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away.").

[0058] Server: Sends the generated answers to the terminals of the field activity team.

[0059] Terminal (Field Team): The answer is displayed to the user.

[0060] 4. Organizing and sharing overall information

[0061] Server: All information is periodically aggregated and AI organizes it in a visually easy-to-understand manner.

[0062] Server: View overall information on the dashboard.

[0063] Server: Sends notifications to all user devices when information is updated.

[0064] Terminals (Control Team and Field Operations Team): View the dashboard and access real-time updated information.

[0065] Specific examples

[0066] For example, if building A collapses during a disaster, an operator from the local activity team can type "Building A collapsed, 10 injured" into chat on their smartphone. This information is sent to the server, where it is analyzed by AI and organized as "Location: Building A, Situation: Collapsed, Number of injured: 10." The organized information is displayed on the control team's dashboard, contributing to the rapid decision-making of restoration instructions.

[0067] Additionally, when a local activity team asks, "Where is the nearest evacuation shelter?" the server checks the database and replies, "It's at the community center, a 10-minute walk away," which is displayed on the local activity team's terminal.

[0068] This allows information to be shared smoothly between the field activity team and the control team, reducing misunderstandings and duplicate questions, and enabling efficient disaster response.

[0069] The processing flow will be explained below.

[0070] Specific processing steps of the program

[0071] User Authentication Process

[0072] Step 1:

[0073] Device: The user launches the app and enters their ID and password on the login screen.

[0074] Step 2:

[0075] Terminal: Sends the entered ID and password to the server.

[0076] Step 3:

[0077] Server: Compares the received ID and password with the database and generates an authentication result.

[0078] Step 4:

[0079] Server: If authentication is successful, generates an authentication token and sends it to the terminal. If authentication fails, sends an error message to the terminal.

[0080] Step 5:

[0081] Terminal: Displays the authentication result (success or failure) to the user.

[0082] Information gathering and interviews

[0083] Step 1:

[0084] User (on-site activity team): Enters the situation at the scene in chat format (e.g., "Building A has collapsed, 10 injured.").

[0085] Step 2:

[0086] Terminal: Sends user input to the server.

[0087] Step 3:

[0088] Server: The received information is analyzed and the AI ​​classifies the content in detail (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10.").

[0089] Step 4:

[0090] Server: Stores organized information in a database.

[0091] Step 5:

[0092] Server: Organized information is displayed in real time on a dashboard.

[0093] Step 6:

[0094] Terminal (control team): Organized information is displayed on a dashboard, allowing the control team to check the latest information.

[0095] Automated response to inquiries

[0096] Step 1:

[0097] User (Field Operations Team): Asks the AI ​​for specific information (e.g., "Where is the nearest evacuation shelter?").

[0098] Step 2:

[0099] Terminal: Sends the query to the server.

[0100] Step 3:

[0101] Server: The AI ​​analyzes the incoming inquiries and compares them with the latest information in the database.

[0102] Step 4:

[0103] Server: Generates an appropriate answer (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away.").

[0104] Step 5:

[0105] Server: Sends the generated answers to the terminals of the field activity team.

[0106] Step 6:

[0107] Terminal (Field Team): Displays the generated answers to the user.

[0108] Organizing and sharing overall information

[0109] Step 1:

[0110] Server: All collected information is periodically aggregated and systematically organized by AI.

[0111] Step 2:

[0112] Server: View organized overview information on the dashboard.

[0113] Step 3:

[0114] Server: Sends real-time notifications to user devices whenever information is updated.

[0115] Step 4:

[0116] Devices (Control and Field Operations Teams): Receive notifications and access the dashboard for the latest information.

[0117] Step 5:

[0118] Terminals (control team and field operation team): Detailed information can be retrieved from the dashboard as needed to help understand the situation on site and carry out rescue operations.

[0119] Example 1

[0120] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0121] There is a problem that information sharing between the field operation team and the control team is not smooth, and it takes a long time to share and analyze information. In addition, it is not possible to respond appropriately to questions and information gathering on the ground in real time, making it difficult to respond to disasters efficiently. Furthermore, there is a lack of visual organization of information and real-time notification of overall information, which makes it difficult for the field and command teams to act quickly.

[0122] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0123] In this invention, the server includes means for receiving information entered by a user and having AI analyze and organize the information, means for storing the organized information in a database and displaying it on a display means, means for the server to periodically aggregate all information and visually display the overall information, means for sending notifications to multiple user devices, means for AI to automatically respond to user inquiries, means for making the overall information accessible from user devices, and means for receiving user authentication information and returning the authentication result to the terminal. This allows information to be shared smoothly between the field activity team and the control team, enabling real-time information provision and response, and achieving efficient disaster response.

[0124] ---

[0125] "User" means a person who uses the system to enter or access information.

[0126] "Server" means a central management system for receiving, analyzing, organizing, storing, displaying, and sending notifications about information.

[0127] "Terminal" refers to a device used by a user, an electronic device that communicates with a server through an application.

[0128] "Information" refers to data such as on-site conditions and inquiries entered by the field activity team and control team.

[0129] "AI" stands for artificial intelligence, and refers to technology that analyzes incoming information, organizes it appropriately, and generates answers.

[0130] A "database" refers to a system for storing organized information and retrieving it as needed.

[0131] "Display means" refers to an interface for visually presenting organized information to a user.

[0132] "Aggregation" refers to the process of bringing together distributed information.

[0133] "Visually displaying" refers to presenting information in a format such as a graph or chart in an easy-to-read format to the user.

[0134] "Notification" refers to a message that informs a user device that new information exists.

[0135] An "inquiry" refers to an action in which a user requests information from a system.

[0136] "Authentication" refers to the process of verifying a user's ID and password and confirming that the user is a legitimate user.

[0137] ---

[0138] ---

[0139] The present invention is a system for facilitating information sharing between the control team and the field activity team. This system consists of terminals used by the control team and the field activity team, and a server that centrally manages information. Below, we will explain in detail how each component works.

[0140] First, a user accesses the system using a terminal. At this time, the user enters their ID and password on the login screen and sends the authentication information to the server. The server compares the received information with the database and returns the authentication result to the terminal. If authentication is successful, the user is allowed to access the dashboard.

[0141] Next, users from the field operation team use their devices to report on the situation at the scene. The users input information in chat format and send it to the server. The server uses AI to analyze the received information and organizes the results of the analysis. For example, if a report is made such as "Building collapsed. 10 injured," the AI ​​will organize this into detailed items such as "Location: Building A, Situation: Collapsed, Number of injured: 10." This organized information is stored in a database and displayed in real time on a dashboard.

[0142] Users can also inquire about the information they need on-site. For example, by asking the AI ​​a question such as "Where is the nearest evacuation shelter?", the AI ​​sends the question to the server. The server then uses the AI ​​to compare the latest information with the database and generate an appropriate answer. The generated answer is sent to the device of the on-site team and displayed to the user.

[0143] The server periodically aggregates all information, and AI organizes it visually for easy understanding. The organized overall information is displayed on a dashboard, and the server sends notifications to all user devices when the information is updated. This allows control teams and field teams to access the latest information in real time and respond quickly.

[0144] Specific examples

[0145] For example, if Building A collapses during a disaster, an operator from the field operation team can enter "Building A collapsed, 10 injured" into the chat app on their smartphone. This information is sent to the server, where the AI ​​analyzes and organizes it as "Location: Building A, Situation: Collapsed, Number of injured: 10," and stores it in a database. On the control team's device, this information is displayed in real time on a dashboard, helping them to quickly decide on recovery instructions.

[0146] In addition, when a field activity team inputs a question into their smartphone, such as "Where is the nearest evacuation shelter?", the server checks the database and generates the answer, "It's at the community center, a 10-minute walk away," which is displayed on the field activity team's device. This allows necessary information to be provided quickly, allowing field activities to proceed efficiently.

[0147] Example prompts for generative AI models

[0148] 1. "The field team reports that Building A has collapsed and 10 people are injured. I want to analyze and organize this information and display it on my dashboard."

[0149] 2. "In response to a question from the field team, 'Where is the nearest evacuation shelter?', please check the database and generate an appropriate answer."

[0150] These prompts facilitate collaboration between the server and the AI, enabling them to effectively utilize input information from the user.

[0151] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0152] Now, let's explain the processing flow of the system program by dividing it into specific processing steps, and add inputs, outputs, and specific actions to each step.

[0153] ---

[0154] Step 1:

[0155] Input: The user launches the app and enters their ID and password on the login screen.

[0156] Specific operation: The user starts the device, opens the app, and accesses the login screen. They enter their ID and password and click the "Login" button.

[0157] Output: The entered ID and password are sent from the terminal to the server.

[0158] Step 2:

[0159] Input: Login information (ID and password)

[0160] Specific operation: The terminal uses a secure protocol (e.g., TLS / SSL) to encrypt the entered ID and password and send them to the server.

[0161] Output: Login information received by the server

[0162] Step 3:

[0163] Input: Login information received by the server

[0164] Specific operation: The server connects to a database (e.g., MySQL (registered trademark), PostgreSQL) and verifies the received ID and password against the registered information in the database. It generates an authentication result and returns the result (success or failure) to the terminal.

[0165] Output: The authentication result is generated and returned to the terminal.

[0166] Step 4:

[0167] Input: Authentication result

[0168] Specific operation: The device receives the authentication result, and if successful, allows the user to access the dashboard. A message indicating authentication success is displayed, and the device transitions to the dashboard screen. If unsuccessful, an error message is displayed.

[0169] Output: User accesses dashboard or is returned to login screen again

[0170] Step 5:

[0171] Input: User (on-site activity team) inputs the situation on-site in chat format

[0172] Specific operations: A user of the field operation team enters on-site information using the chat interface on the device and clicks the "Send" button.

[0173] Output: The entered site conditions are sent from the terminal to the server.

[0174] Step 6:

[0175] Input: Site status information

[0176] Specific operation: The server uses AI (natural language processing technology) to analyze the received information and organize it into specific categories (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10."). The organized information is then stored in a database.

[0177] Output: The analyzed and organized information is saved in a database.

[0178] Step 7:

[0179] Input: Organized information

[0180] Specific operation: The server prepares the organized information for display on the dashboard and updates the dashboard.

[0181] Output: The dashboard is updated to show organized information

[0182] Step 8:

[0183] Input: User (on-site activity team) enters the inquiry details

[0184] Specific operation: A user from the field team uses the chat interface on the device to enter a question and press the "send" button.

[0185] Output: The query is sent to the server

[0186] Step 9:

[0187] Input: Inquiry details

[0188] How it works: The server uses AI to analyze the received inquiry and compares it with the database and the latest information. An appropriate answer is generated (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away."). The answer is then sent to the device.

[0189] Output: The generated answer is sent to the device.

[0190] Step 10:

[0191] Input: Answer

[0192] Specific behavior: The device displays the received response in the chat interface within the app.

[0193] Output: The answer is displayed to the user

[0194] Step 11:

[0195] Input: General information updated regularly

[0196] How it works: The server periodically aggregates all information, and AI organizes it visually for easy understanding. The organized information is displayed on a dashboard, and notifications are sent to all user devices when the information is updated.

[0197] Output: Overall information is displayed on the dashboard and notifications are sent when information is updated.

[0198] Step 12:

[0199] Input: Information update notification

[0200] What happens: All user devices receive notifications and access the dashboard.

[0201] Output: User accesses the latest global information

[0202] This will allow you to understand the processing flow of the entire system in concrete detail.

[0203] (Application example 1)

[0204] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0205] There is a problem with insufficient information sharing between the field activity team and the control team during emergencies, making it difficult to efficiently manage and coordinate information in real time. In particular, there is a need for a system that can respond quickly and accurately to reports and inquiries from the field and visually grasp the overall situation.

[0206] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0207] In this invention, the server includes means for receiving information entered by the field activity team and having an AI analyze and organize the information, means for storing the organized information in a database and displaying it on a dashboard, means for the server to periodically aggregate all information and visually display overall information, means for sending notifications to multiple user devices, means for the AI ​​to automatically respond to inquiries from the field activity team, means for making overall information accessible from user devices, means for user authentication, means for reporting emergencies in chat format, means for visually displaying analyzed information, means for using an AI model to compare data and generate responses in an emergency, means for transmitting the generated responses to user devices, and means for updating and displaying information on user devices in real time. This allows information to be shared quickly in real time between the field activity team and the control team, enabling efficient emergency response.

[0208] The "on-site activity team" is a team that actually collects information at the scene of an emergency or disaster and reports on it.

[0209] The "control team" is a central management team that issues instructions based on information provided by the field operation team.

[0210] "AI" is an abbreviation for artificial intelligence, a technology that analyzes data and automatically makes judgments and predictions based on the results.

[0211] A "server" is a computer system that centrally manages information and executes various processes.

[0212] "User devices" are terminals such as smartphones and tablets used by field teams and control teams.

[0213] A "dashboard" is a screen or interface that visually displays information and allows users to grasp the situation in real time.

[0214] A "database" is an information storage system that stores organized information and allows it to be searched and used as needed.

[0215] "Notifications" is a feature that sends information updates and important announcements to user devices.

[0216] "User authentication" is a procedure for verifying that a user is a legitimate user.

[0217] "Chat format" is a text-based communication method that allows for real-time conversation.

[0218] "Analysis" is the process of analyzing information and organizing it into a meaningful form.

[0219] "Visual display" means displaying information in an easy-to-understand manner using graphs, maps, etc.

[0220] "Data matching" is the process of comparing input information with existing data to see if there is a match.

[0221] "Answer generation" is a function that automatically creates an appropriate response based on the results of data matching.

[0222] "Real-time" means that information is processed and notified within a very short time after it is generated.

[0223] This invention is a system that allows field teams and control teams to share information quickly and accurately in an emergency. The system receives information entered by the field team and uses AI to analyze and organize it, allowing the control team to grasp the situation in real time.

[0224] The system mainly consists of a server and a user device.

[0225] Hardware and Software

[0226] Hardware:

[0227] Server (Central Information Management System)

[0228] Smartphones (devices for the field activity team and control team)

[0229] software:

[0230] Flask (Python framework)

[0231] AI module (for data analysis)

[0232] Server Processing

[0233] The server receives information reported by the field teams, which are then analyzed by AI. The analyzed information is organized into a visually easy-to-understand format and stored in a database. It is then displayed in real time on a dashboard. The AI ​​also automatically responds to inquiries from the field teams, generating appropriate answers and sending them to the field teams' user devices.

[0234] User Device Capabilities

[0235] The user devices are smartphones used by the field operation team and the control team. These devices perform user authentication and have the function of reporting emergencies in chat format. The reported information is sent to a server, and the analysis results are displayed on the user device in real time.

[0236] Specific examples

[0237] For example, if a field team reports using their smartphone that "Building B is on fire, 5 injured," the information is analyzed by the server and displayed in organized form as "Location: Building B, Situation: Fire, Number of injured: 5." The control team can view the dashboard and issue instructions quickly based on the latest information. Also, if the field team asks, "Where is the nearest evacuation shelter?" the AI ​​will check the database and answer, "The nearest evacuation shelter is the community center, a 10-minute walk away," and send this information to the field team's device.

[0238] Prompt Sentence Examples

[0239] "As an AI model for a safety management system, please analyze the following information: Building B is on fire, 5 injured."

[0240] In this way, information management and sharing during emergencies can be carried out quickly and accurately, enabling efficient responses.

[0241] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0242] Step 1:

[0243] The user starts up their smartphone device and enters their ID and password on the login screen. The device sends the entered ID and password to the server. The server checks the received user information in a database and returns the authentication result to the device. The input is the user ID and password, and the output is the authentication result. Data processing involves verifying the user information, thereby confirming that the user is legitimate.

[0244] Step 2:

[0245] The user (on-site action team) reports the status of the emergency in chat format. Specifically, they input text information such as "Building B is on fire, 5 injured." The device sends this report to the server. The server then gives the received information to the AI ​​for analysis. The input is the emergency report, and the output is the analyzed specific situation information. Data processing involves text analysis, which organizes the information.

[0246] Step 3:

[0247] The server stores the information analyzed by AI in a database. The organized information is then displayed in real time on a dashboard. The input is the information analyzed by AI, and the output is the information stored in the database and displayed on the dashboard. Data processing involves storing and visualizing the information, making it usable.

[0248] Step 4:

[0249] The local activity team queries the AI ​​for the necessary information. Specifically, they input the question, "Where is the nearest evacuation shelter?" The device sends this query to the server. The server's AI compares the latest information with the database and generates an appropriate answer. The input is the query, and the output is the generated answer. Data processing involves data comparison and answer generation, which provides appropriate information.

[0250] Step 5:

[0251] The server sends the generated response to the terminal of the field team, who then checks the response on their terminal. The input is the response information from the server, and the output is the response displayed on the user's terminal. Data processing involves sending and displaying the information, which allows the field team to obtain the necessary information.

[0252] Step 6:

[0253] The server periodically aggregates all information, and AI organizes it in a visually easy-to-understand format. Specifically, the overall information is displayed in the form of graphs and maps. The input is individual pieces of information data, and the output is overall information that has been visually organized. Data processing involves aggregating and visualizing the information, which allows the overall situation to be grasped.

[0254] Step 7:

[0255] Every time the information is updated, the server sends notifications to all user devices in real time. The input is the new updated information, and the output is the notification to the user device. The data processing is the notification generation and transmission, so that users receive the latest information in real time.

[0256] In this way, by specifically showing how input data is processed at each step and what output is obtained, it becomes easier to understand the flow of the entire system.

[0257] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0258] This invention is a system for smooth information sharing between the control team and the field activity team, and is combined with an emotion engine that recognizes the user's emotions. This system consists of devices used by the control team and the field activity team, and a central server that manages information. The server uses AI and the emotion engine to receive, analyze, organize, and share information as appropriate.

[0259] Program processing flow (natural language explanation)

[0260] 1. User Authentication

[0261] Device: The user launches the app and enters their ID and password on the login screen.

[0262] Terminal: Sends the entered ID and password to the server.

[0263] Server: Checks the received user information against the database and generates an authentication result.

[0264] Server: If authentication is successful, generates an authentication token and sends it to the terminal. If authentication fails, sends an error message to the terminal.

[0265] Terminal: Displays the authentication result (success or failure) to the user.

[0266] 2. Information gathering and interviews

[0267] User (on-site activity team): Enters the situation at the scene in chat format (e.g., "Building A has collapsed, 10 injured.").

[0268] Terminal: Sends user input to the server.

[0269] Server: The received information is analyzed and the AI ​​classifies the content in detail (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10.").

[0270] Server: Stores organized information in a database.

[0271] Server: Organized information is displayed in real time on a dashboard.

[0272] Terminal (control team): Organized information is displayed on a dashboard, allowing the control team to check the latest information.

[0273] 3. Automated response to inquiries

[0274] User (Field Operations Team): Asks the AI ​​for specific information (e.g., "Where is the nearest evacuation shelter?").

[0275] Terminal: Sends the query to the server.

[0276] Server: The AI ​​analyzes the incoming inquiries and compares them with the latest information in the database.

[0277] Server: Generates an appropriate answer (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away.").

[0278] Server: Sends the generated answers to the terminals of the field activity team.

[0279] Terminal (Field Team): Displays the generated answers to the user.

[0280] 4. Organizing and sharing overall information

[0281] Server: All collected information is periodically aggregated and systematically organized by AI.

[0282] Server: View organized overview information on the dashboard.

[0283] Server: Sends real-time notifications to user devices whenever information is updated.

[0284] Devices (Control and Field Operations Teams): Receive notifications and access the dashboard for the latest information.

[0285] Terminals (control team and field operation team): Detailed information can be retrieved from the dashboard as needed to help understand the situation on site and carry out rescue operations.

[0286] 5. Recognizing and responding to user emotions

[0287] User (field team or control team): Expresses emotions (e.g., tension, anger) when interacting with the system through tone of voice and input text.

[0288] Terminal: Sends the user's emotional information to the server.

[0289] Server: The emotion engine analyzes the user's tone of voice and text to identify emotions (e.g., if the voice is trembling, it is recognized as nervousness).

[0290] Server: Generates an appropriate response based on the identified emotion (e.g., a calming message to ease tension).

[0291] Server: Sends the generated response to the user device and displays it to the user.

[0292] Specific examples

[0293] For example, if building A collapses during a disaster, an operator from the local activity team can type "Building A collapsed, 10 injured" into the chat on their smartphone. This information is sent to the server, where it is analyzed by AI and organized as "Location: Building A, Situation: Collapsed, Number of injured: 10." The organized information is displayed on the control team's dashboard, contributing to the prompt issuing of recovery instructions.

[0294] Furthermore, when a field team asks, "Where is the nearest evacuation shelter?", the server checks the database and replies, "It's at the community center, a 10-minute walk away," which is displayed on the field team's terminal. If the operator's voice is trembling, the emotion engine recognizes this as "tension" and automatically generates a response such as, "Please stay calm, the evacuation shelter is nearby."

[0295] This allows for smooth sharing of information between the field activity team and the control team, reducing misunderstandings and duplicate questions, enabling efficient disaster response, and also responding to users' emotions.

[0296] The processing flow will be explained below.

[0297] Specific processing steps of the program

[0298] User Authentication Process

[0299] Step 1:

[0300] Device: The user launches the app and enters their ID and password on the login screen.

[0301] Step 2:

[0302] Terminal: Sends the entered ID and password to the server.

[0303] Step 3:

[0304] Server: Compares the received ID and password with the database and generates an authentication result.

[0305] Step 4:

[0306] Server: If authentication is successful, generates an authentication token and sends it to the terminal. If authentication fails, sends an error message to the terminal.

[0307] Step 5:

[0308] Terminal: Displays the authentication result (success or failure) to the user.

[0309] Information gathering and interviews

[0310] Step 1:

[0311] User (on-site activity team): Enters the situation at the scene in chat format (e.g., "Building A has collapsed, 10 injured.").

[0312] Step 2:

[0313] Terminal: Sends user input to the server.

[0314] Step 3:

[0315] Server: The received information is analyzed and the AI ​​classifies the content in detail (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10.").

[0316] Step 4:

[0317] Server: Stores organized information in a database.

[0318] Step 5:

[0319] Server: Organized information is displayed in real time on a dashboard.

[0320] Step 6:

[0321] Terminal (control team): Organized information is displayed on a dashboard, allowing the control team to check the latest information.

[0322] Automated response to inquiries

[0323] Step 1:

[0324] User (Field Operations Team): Asks the AI ​​for specific information (e.g., "Where is the nearest evacuation shelter?").

[0325] Step 2:

[0326] Terminal: Sends the query to the server.

[0327] Step 3:

[0328] Server: The AI ​​analyzes the incoming inquiries and compares them with the latest information in the database.

[0329] Step 4:

[0330] Server: Generates an appropriate answer (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away.").

[0331] Step 5:

[0332] Server: Sends the generated answers to the terminals of the field activity team.

[0333] Step 6:

[0334] Terminal (Field Team): Displays the generated answers to the user.

[0335] Organizing and sharing overall information

[0336] Step 1:

[0337] Server: All collected information is periodically aggregated and systematically organized by AI.

[0338] Step 2:

[0339] Server: View organized overview information on the dashboard.

[0340] Step 3:

[0341] Server: Sends real-time notifications to user devices whenever information is updated.

[0342] Step 4:

[0343] Devices (Control and Field Operations Teams): Receive notifications and access the dashboard for the latest information.

[0344] Step 5:

[0345] Terminals (control team and field operation team): Detailed information can be retrieved from the dashboard as needed to help understand the situation on site and carry out rescue operations.

[0346] Recognizing and responding to user emotions

[0347] Step 1:

[0348] Users (field or control team): Express emotions (e.g., tension, anger) when interacting with the system through tone of voice and input text.

[0349] Step 2:

[0350] Terminal: Sends the user's emotional information to the server.

[0351] Step 3:

[0352] Server: The emotion engine analyzes the user's tone of voice and text to identify emotions (e.g., if the voice is trembling, it is recognized as nervousness).

[0353] Step 4:

[0354] Server: Generates an appropriate response based on the identified emotion (e.g., a calming message to ease tension).

[0355] Step 5:

[0356] Server: Sends the generated response to the user device and displays it to the user.

[0357] Example 2

[0358] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0359] When coordinating information between the field operation team and the control team, it is difficult to share accurate and appropriate information in real time. In addition, there is a need to recognize the emotions of users working in the field and respond appropriately, but current systems do not adequately fulfill this requirement.

[0360] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0361] In this invention, the server includes means for receiving information entered by the field activity team and analyzing and organizing the information using a machine learning model, means for storing the organized information in a database and displaying it on an information display device, means for a management device to periodically aggregate all information and visually display overall information, means for sending notifications to multiple user terminals, means for a generation engine to automatically respond to inquiries from the field activity team, means for analyzing tone of voice and input text using an emotion engine that recognizes user emotions and generating an appropriate response message, and means for making overall information accessible from user terminals. This enables accurate and appropriate information sharing in real time between the field activity team and the control team, and realizes flexible responses according to user emotions.

[0362] "Field activity teams" are personnel or teams that mainly operate on-site, reporting on the situation and gathering information.

[0363] "Machine learning models" refer to the algorithms and mathematical models used by the server to perform data analysis and pattern recognition.

[0364] "Means of organizing" refers to the function of receiving and analyzing information, classifying and arranging the content in a specific format, and storing and displaying it.

[0365] A "database" is an information system designed to efficiently store and manage organized information and enable quick retrieval when needed.

[0366] "Information display device" refers to equipment or software that provides an interface for visually checking and manipulating collected information.

[0367] "Management device" refers to a computer system that centrally manages the collection, organization, and display of all information.

[0368] "User terminal" refers to a device (e.g., smartphone, tablet, PC) used by the field activity team and control team that can input and output information.

[0369] "Means for sending notifications" refers to a function that allows the server to notify the user terminal of information updates and important events in real time.

[0370] "Generation engine" refers to AI technology that automatically generates appropriate answers to user inquiries.

[0371] An "emotion engine" refers to AI technology that analyzes emotions from the user's tone of voice and input text, and automatically generates a response appropriate to those emotions.

[0372] "Comprehensive information" refers to comprehensive information that integrates and organizes all data collected from field teams and other sources.

[0373] "Visual display means" refers to the function of displaying collected and organized information in the form of graphs, charts, text, etc. in a way that is easy to understand at a glance.

[0374] "Means of making it accessible" refers to the mechanism that provides the authority and functions for a user device to view and operate the entire information.

[0375] This invention is a system for facilitating information sharing between field teams and control teams, and is combined with an emotion engine that recognizes user emotions. This system consists of devices used by the field teams and control teams, and a central server that manages information. The server utilizes machine learning models and the emotion engine to receive, analyze, organize, share appropriate information, and respond to emotions.

[0376] The server is equipped with a database, machine learning model, and emotion engine. PostgreSQL is used as the database, and TENSORFLOW (registered trademark) is used as the machine learning model. IBM Watson (registered trademark) Tone Analyzer is used as the emotion engine. In addition, Grafana is used as the information display device, and Firebase Cloud Messaging (FCM) is used for the notification function.

[0377] Users of the field response team interactively input the situation at the scene. This information is sent to the server via their terminal. The server analyzes the received information using a machine learning model and categorizes and organizes it in detail. For example, an input such as "Building A collapsed, 10 injured" is organized as "Location: Building A, Situation: Collapsed, Number of injured: 10." The organized information is stored in a PostgreSQL database.

[0378] The server then displays the organized information in real time on the Grafana dashboard via the management device, allowing the control team to view the latest information on their terminals. At the same time, the server periodically aggregates all the information and provides a means for visually displaying it. Every time this information is updated, the server sends a real-time notification to the user's terminal.

[0379] Additionally, when the field team makes a direct inquiry to the server, such as "Where is the nearest evacuation shelter?", the server automatically generates an appropriate answer using a generative AI model (OpenAI® GPT-4® model). The answer automatically generated is "It's a 10-minute walk from the community center," and is displayed on the field team's terminal.

[0380] Furthermore, an emotion engine (IBM Watson Tone Analyzer) analyzes emotions expressed by the user while interacting with the system, such as tension or anger, and automatically generates an appropriate response message. If the user is nervous, a message such as "Please stay calm, a shelter is nearby" is generated and sent to the user's device.

[0381] For example, when a disaster occurs, an operator from the field team can type "Building A has collapsed, 10 injured" into the chat on their smartphone. This information is analyzed and organized on the server and displayed on the control team's dashboard. If the field team asks, "Where is the nearest evacuation shelter?", the server will respond with "The community center, a 10-minute walk away." At the same time, if the operator is nervous, a message will be sent saying, "Please stay calm, the evacuation shelter is nearby."

[0382] Examples of prompts are:

[0383] "Building A has collapsed. 10 people are injured. Please tell me the nearest evacuation shelter."

[0384] "I want to see the current status of injured people in real time. How can I access the dashboard?"

[0385] "What kind of message would be effective for a user who is nervous?"

[0386] This enables accurate and appropriate information sharing in real time between the field operation team and the control team, and also enables flexible responses to user emotions.

[0387] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0388] Program processing flow and specific explanation of each processing step

[0389] Step 1: User authentication

[0390] 1. Input: The user enters the ID and password into the terminal.

[0391] 2. Operation: The device hashes the entered ID and password and sends the data to the server via secure communication (HTTPS).

[0392] 3. Server:

[0393] Data processing: The server receives the hashed user information and checks it against the MySQL database.

[0394] Data operation: Executes SQL queries to retrieve user information and generate authentication results.

[0395] 4. Output: If authentication is successful, a JWT (JSON Web Token) is generated and sent to the terminal. If authentication fails, an error message is generated and sent.

[0396] Step 2: Gathering information and conducting interviews

[0397] 1. Input: The user (on-site activity team) inputs the situation on the scene in chat format. Example: "Building A collapsed, 10 injured."

[0398] 2. Operation: The device sends the input information to the server via HTTPS.

[0399] 3. Server:

[0400] Data processing: The text information received by the server is analyzed using TensorFlow's machine learning model and classified in detail.

[0401] Data calculation: Classify the information as "Location: Building A, Situation: Collapsed, Number of injured: 10."

[0402] 4. Output: The classified information is stored in a PostgreSQL database and sent to an information display device (Grafana) in real time for display.

[0403] Step 3: Automate your inquiries

[0404] 1. Input: A user (field operations team) asks for specific information, e.g., "Where is the nearest evacuation shelter?"

[0405] 2. Operation: The device sends the query to the server via HTTPS.

[0406] 3. Server:

[0407] Data processing: The query received by the server is input into the OpenAI GPT-4 model for analysis.

[0408] Data calculations: AI compares databases with the latest information and generates appropriate answers.

[0409] 4. Output: The generated answer is sent to the device and displayed to the user. For example, "The nearest evacuation shelter is the Civic Center, a 10-minute walk away."

[0410] Step 4: Organize and share overall information

[0411] 1. Input: All collected information is collected on the server.

[0412] 2. Operation: The server periodically aggregates this information.

[0413] 3. Server:

[0414] Data processing: Information is systematically organized using deep learning models running on NVIDIA GPUs with CUDA acceleration.

[0415] Data calculation: Visually organize the overall information.

[0416] 4. Output: The organized overall information is displayed on a Grafana dashboard, and notifications are sent to the user's device using Firebase Cloud Messaging (FCM) every time the information is updated.

[0417] Step 5: Recognize and respond to user emotions

[0418] 1. Input: The user expresses their emotions through tone of voice and text while interacting with the system.

[0419] 2. Operation: The device sends voice and text information expressing emotions to the server via WebRTC.

[0420] 3. Server:

[0421] Data processing: The emotion engine (IBM Watson Tone Analyzer) analyzes the received voice and text to identify emotions.

[0422] Data Calculation: Based on the analysis results, an appropriate response message is generated.

[0423] 4. Output: Send the generated message to the terminal and display it to the user. Example: "Please stay calm, shelter is nearby."

[0424] This allows the system to process information quickly and accurately at each stage, enabling real-time information sharing between the field operation team and the control team, while also enabling it to flexibly respond to user emotions.

[0425] (Application example 2)

[0426] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0427] Conventional information sharing systems for on-site work have had issues such as time-consuming information organization and analysis, and difficulty in responding immediately in emergencies. Furthermore, the lack of feedback that takes into account the emotions and mental state of workers makes it difficult to create an efficient and safe work environment. This invention solves these issues by providing rapid and accurate information sharing and feedback that also takes into account the mental state of workers.

[0428] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving information entered by the field activity team and analyzing and organizing the information using AI, means for storing the organized information in a database and displaying it on a dashboard, and means for periodically aggregating information and visually displaying the overall information. This enables rapid organization and sharing of information. In addition, by using means for recognizing user emotions and providing appropriate feedback, a safe and efficient work environment for workers is ensured.

[0429] A "field unit" is a person or team whose role is to gather and report information at a specific site.

[0430] "AI" refers to the intelligent behavior and learning capabilities imitated by computer systems as artificial intelligence.

[0431] A "dashboard" is an interface for visually displaying collected information, allowing users to check the situation in real time.

[0432] "Server" means a central management system that receives, analyzes, stores, and notifies user devices of information.

[0433] "User device" refers to a terminal used by field teams and control teams, including smartphones, tablets, smart glasses, etc.

[0434] "Sending notifications" refers to the act of automatically sending updates or responses from the server to the user device.

[0435] An "emotion engine" is a technology that recognizes emotions by analyzing the user's tone of voice and input text.

[0436] "Feedback" is the act of providing appropriate responses or instructions according to the user's emotions and situation.

[0437] "Analyzing and organizing information" means that the AI ​​processes, classifies, and reconstructs the information entered by the field team into an easy-to-understand format.

[0438] This invention is a system that enables smooth information sharing between field teams and control teams, and combines an emotion engine that recognizes the user's emotions. The field teams are responsible for collecting and reporting information at specific sites, while the control teams are responsible for managing information centrally and issuing instructions.

[0439] Hardware and software used

[0440] Hardware:

[0441] Smart glasses (e.g., Microsoft® HoloLens®)

[0442] Server (e.g. AWS (registered trademark) EC2)

[0443] software:

[0444] AI analysis engine (e.g. TensorFlow)

[0445] Emotion recognition engine (e.g. IBM Watson Tone Analyzer)

[0446] Database (e.g. MySQL)

[0447] Front-end frameworks (e.g., React.js)

[0448] Backend frameworks (e.g., Node.js)

[0449] Explanation of the generated program

[0450] 1. User authentication:

[0451] Terminal: The user turns on the smart glasses and enters their ID and password. The entered information is sent to the server.

[0452] Server: Checks the received user information in the database and generates an authentication result. If authentication is successful, generates a token and sends it to the terminal.

[0453] Terminal: Displays the authentication result to the user.

[0454] 2. Information gathering and interviews:

[0455] User (field operation team): Collects on-site information using the smart glasses' camera and voice input (e.g., "The conveyor belt has stopped").

[0456] Terminal: Sends the entered information to the server.

[0457] Server: The AI ​​analysis engine processes the received information and classifies it in detail.

[0458] Server: Stores organized information in a database and displays it on a dashboard.

[0459] 3. Automated response to inquiries:

[0460] User (Field Team): Asks the AI ​​for specific information via smart glasses (e.g., "Where is the manual for this device?").

[0461] Terminal: Sends the query to the server.

[0462] Server: The AI ​​analysis engine analyzes the received inquiry, compares it with the database and the latest information, and generates an appropriate answer.

[0463] Server: Sends the generated answer to the user's terminal and displays it to the user.

[0464] 4. Emotion Recognition and Feedback:

[0465] User (field team or control team): Expresses emotions through tone of voice and text input.

[0466] Terminal: Sends emotion information to the server.

[0467] Server: The emotion recognition engine analyzes this and identifies the emotion.

[0468] Server: Generates appropriate feedback, sends it to the user's device, and displays it (e.g., "Please stay calm, our support team will assist you").

[0469] Specific examples

[0470] For example, if a problem occurs in a factory where the conveyor belt has stopped, a worker on the field team can use smart glasses to capture the situation with a camera and report the situation by voice, saying, "The conveyor belt has stopped." This information is sent to the server, where the AI ​​analysis engine analyzes it and classifies it as "Location: Conveyor belt, Status: Stopped," and displays it on a dashboard. The control team can then quickly issue instructions by looking at the dashboard. Furthermore, if the worker's trembling voice is identified as "tension" by the emotion recognition engine, feedback such as "Please stay calm and we will contact the technical support team" is automatically returned.

[0471] Example prompt sentence:

[0472] "A worker in a factory uses smart glasses to report a machine problem. Write the steps to resolve this problem. Also, include an action that recognizes the worker's emotions and provides appropriate feedback."

[0473] This allows the field team and the control team to share information quickly and accurately and take immediate action.It also responds to the emotions of workers, providing a safer and more efficient working environment.

[0474] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0475] Step 1: User authentication

[0476] Input: The user turns on the smart glasses and enters their ID and password.

[0477] Terminal operation: The entered ID and password are sent to the server.

[0478] Server operation: The received user information is checked against the database, and an authentication result is generated. If authentication is successful, an authentication token is generated and sent to the terminal.

[0479] Output: The terminal displays the authentication result to the user.

[0480] Step 2: Information gathering and interviews

[0481] Input: The user (field team) takes pictures of the scene with the camera on the smart glasses and inputs information by voice. For example, they report that the conveyor belt has stopped.

[0482] Device operation: Sends the user's voice input and camera images to the server.

[0483] Server operation: The AI ​​analysis engine processes the received information and classifies it in detail. For example, "Location: conveyor belt, status: stopped."

[0484] Output: The server stores the organized information in a database and displays it on a dashboard.

[0485] Step 3: Automate your inquiries

[0486] Input: A user (field team) queries the AI ​​for specific information via smart glasses, for example, "Where is the manual for this device?"

[0487] Terminal operation: Sends the user's inquiry to the server.

[0488] Server operation: The AI ​​analysis engine analyzes the received inquiry, compares it with the database and the latest information, and generates an appropriate answer.

[0489] Output: The server sends the generated answer to the field team's terminal and displays it to the user.

[0490] Step 4: Emotion recognition and feedback

[0491] Input: Users (field or control teams) express emotions through tone of voice or text input.

[0492] Device operation: Sends user's voice and text information to the server.

[0493] Server operation: The emotion recognition engine analyzes this and identifies the user's emotion. For example, it recognizes a trembling voice as "tension."

[0494] Output: The server generates appropriate feedback (e.g., "Please stay calm, we will contact our technical support team") and sends it to the user's device for display.

[0495] Step 5: Organize and share the overall information

[0496] Input: All information aggregated on the server.

[0497] Server operation: The server periodically aggregates all information, and the AI ​​analysis engine systematically organizes it.

[0498] Output: The organized information is visually displayed on a dashboard, and real-time notifications are sent to the user's device whenever the information is updated.

[0499] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0500] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0501] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0502] [Second embodiment]

[0503] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0504] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0505] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0506] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0507] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0508] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0509] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0510] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0511] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0512] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0513] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0514] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0515] The present invention is a system for facilitating information sharing between the control team and the field activity team. The program processing in this system will be explained below.

[0516] The system consists of devices used by the control team and field teams, and a central server that manages information. The server uses AI to receive, analyze, organize, and share appropriate information. The system also sends real-time notifications to user devices, ensuring that users always have access to the latest information.

[0517] Program processing flow (natural language explanation)

[0518] 1. User Authentication

[0519] Device: The user launches the app and enters their ID and password on the login screen.

[0520] Terminal: Sends the entered ID and password to the server.

[0521] Server: Checks the received user information in the database and returns the authentication result to the terminal.

[0522] Terminal: If authentication is successful, grant the user access to the dashboard.

[0523] 2. Information gathering and interviews

[0524] User (on-site action team): Reports the situation on-site in chat format (e.g., "The building has collapsed. 10 people injured.").

[0525] Terminal: Sends user input to the server.

[0526] Server: Receives information, and the AI ​​analyzes and organizes it into detailed items (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10.").

[0527] Server: Stores organized information in a database and displays it on a dashboard.

[0528] Terminal (control team): Organized information is displayed in real time on the dashboard.

[0529] 3. Automated response to inquiries

[0530] User (on-site response team): Asks the AI ​​for the necessary information (e.g., "Where is the nearest evacuation shelter?").

[0531] Terminal: Sends the query to the server.

[0532] Server: The AI ​​compares the database with the latest information and generates an appropriate answer (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away.").

[0533] Server: Sends the generated answers to the terminals of the field activity team.

[0534] Terminal (Field Team): The answer is displayed to the user.

[0535] 4. Organizing and sharing overall information

[0536] Server: All information is periodically aggregated and AI organizes it in a visually easy-to-understand manner.

[0537] Server: View overall information on the dashboard.

[0538] Server: Sends notifications to all user devices when information is updated.

[0539] Terminals (Control Team and Field Operations Team): View the dashboard and access real-time updated information.

[0540] Specific examples

[0541] For example, if building A collapses during a disaster, an operator from the local activity team can type "Building A collapsed, 10 injured" into chat on their smartphone. This information is sent to the server, where it is analyzed by AI and organized as "Location: Building A, Situation: Collapsed, Number of injured: 10." The organized information is displayed on the control team's dashboard, contributing to the rapid decision-making of restoration instructions.

[0542] Additionally, when a local activity team asks, "Where is the nearest evacuation shelter?" the server checks the database and replies, "It's at the community center, a 10-minute walk away," which is displayed on the local activity team's terminal.

[0543] This allows information to be shared smoothly between the field activity team and the control team, reducing misunderstandings and duplicate questions, and enabling efficient disaster response.

[0544] The processing flow will be explained below.

[0545] Specific processing steps of the program

[0546] User Authentication Process

[0547] Step 1:

[0548] Device: The user launches the app and enters their ID and password on the login screen.

[0549] Step 2:

[0550] Terminal: Sends the entered ID and password to the server.

[0551] Step 3:

[0552] Server: Compares the received ID and password with the database and generates an authentication result.

[0553] Step 4:

[0554] Server: If authentication is successful, generates an authentication token and sends it to the terminal. If authentication fails, sends an error message to the terminal.

[0555] Step 5:

[0556] Terminal: Displays the authentication result (success or failure) to the user.

[0557] Information gathering and interviews

[0558] Step 1:

[0559] User (on-site activity team): Enters the situation at the scene in chat format (e.g., "Building A has collapsed, 10 injured.").

[0560] Step 2:

[0561] Terminal: Sends user input to the server.

[0562] Step 3:

[0563] Server: The received information is analyzed and the AI ​​classifies the content in detail (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10.").

[0564] Step 4:

[0565] Server: Stores organized information in a database.

[0566] Step 5:

[0567] Server: Organized information is displayed in real time on a dashboard.

[0568] Step 6:

[0569] Terminal (control team): Organized information is displayed on a dashboard, allowing the control team to check the latest information.

[0570] Automated response to inquiries

[0571] Step 1:

[0572] User (Field Operations Team): Asks the AI ​​for specific information (e.g., "Where is the nearest evacuation shelter?").

[0573] Step 2:

[0574] Terminal: Sends the query to the server.

[0575] Step 3:

[0576] Server: The AI ​​analyzes the incoming inquiries and compares them with the latest information in the database.

[0577] Step 4:

[0578] Server: Generates an appropriate answer (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away.").

[0579] Step 5:

[0580] Server: Sends the generated answers to the terminals of the field activity team.

[0581] Step 6:

[0582] Terminal (Field Team): Displays the generated answers to the user.

[0583] Organizing and sharing overall information

[0584] Step 1:

[0585] Server: All collected information is periodically aggregated and systematically organized by AI.

[0586] Step 2:

[0587] Server: View organized overview information on the dashboard.

[0588] Step 3:

[0589] Server: Sends real-time notifications to user devices whenever information is updated.

[0590] Step 4:

[0591] Devices (Control and Field Operations Teams): Receive notifications and access the dashboard for the latest information.

[0592] Step 5:

[0593] Terminals (control team and field operation team): Detailed information can be retrieved from the dashboard as needed to help understand the situation on site and carry out rescue operations.

[0594] Example 1

[0595] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0596] There is a problem that information sharing between the field operation team and the control team is not smooth, and it takes a long time to share and analyze information. In addition, it is not possible to respond appropriately to questions and information gathering on the ground in real time, making it difficult to respond to disasters efficiently. Furthermore, there is a lack of visual organization of information and real-time notification of overall information, which makes it difficult for the field and command teams to act quickly.

[0597] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0598] In this invention, the server includes means for receiving information entered by a user and having AI analyze and organize the information, means for storing the organized information in a database and displaying it on a display means, means for the server to periodically aggregate all information and visually display the overall information, means for sending notifications to multiple user devices, means for AI to automatically respond to user inquiries, means for making the overall information accessible from user devices, and means for receiving user authentication information and returning the authentication result to the terminal. This allows information to be shared smoothly between the field activity team and the control team, enabling real-time information provision and response, and achieving efficient disaster response.

[0599] ---

[0600] "User" means a person who uses the system to enter or access information.

[0601] "Server" means a central management system for receiving, analyzing, organizing, storing, displaying, and sending notifications about information.

[0602] "Terminal" refers to a device used by a user, an electronic device that communicates with a server through an application.

[0603] "Information" refers to data such as on-site conditions and inquiries entered by the field activity team and control team.

[0604] "AI" stands for artificial intelligence, and refers to technology that analyzes incoming information, organizes it appropriately, and generates answers.

[0605] A "database" refers to a system for storing organized information and retrieving it as needed.

[0606] "Display means" refers to an interface for visually presenting organized information to a user.

[0607] "Aggregation" refers to the process of bringing together distributed information.

[0608] "Visually displaying" refers to presenting information in a format such as a graph or chart in an easy-to-read format to the user.

[0609] "Notification" refers to a message that informs a user device that new information exists.

[0610] An "inquiry" refers to an action in which a user requests information from a system.

[0611] "Authentication" refers to the process of verifying a user's ID and password and confirming that the user is a legitimate user.

[0612] ---

[0613] ---

[0614] The present invention is a system for facilitating information sharing between the control team and the field activity team. This system consists of terminals used by the control team and the field activity team, and a server that centrally manages information. Below, we will explain in detail how each component works.

[0615] First, a user accesses the system using a terminal. At this time, the user enters their ID and password on the login screen and sends the authentication information to the server. The server compares the received information with the database and returns the authentication result to the terminal. If authentication is successful, the user is allowed to access the dashboard.

[0616] Next, users from the field operation team use their devices to report on the situation at the scene. The users input information in chat format and send it to the server. The server uses AI to analyze the received information and organizes the results of the analysis. For example, if a report is made such as "Building collapsed. 10 injured," the AI ​​will organize this into detailed items such as "Location: Building A, Situation: Collapsed, Number of injured: 10." This organized information is stored in a database and displayed in real time on a dashboard.

[0617] Users can also inquire about the information they need on-site. For example, by asking the AI ​​a question such as "Where is the nearest evacuation shelter?", the AI ​​sends the question to the server. The server then uses the AI ​​to compare the latest information with the database and generate an appropriate answer. The generated answer is sent to the device of the on-site team and displayed to the user.

[0618] The server periodically aggregates all information, and AI organizes it visually for easy understanding. The organized overall information is displayed on a dashboard, and the server sends notifications to all user devices when the information is updated. This allows control teams and field teams to access the latest information in real time and respond quickly.

[0619] Specific examples

[0620] For example, if Building A collapses during a disaster, an operator from the field operation team can enter "Building A collapsed, 10 injured" into the chat app on their smartphone. This information is sent to the server, where the AI ​​analyzes and organizes it as "Location: Building A, Situation: Collapsed, Number of injured: 10," and stores it in a database. On the control team's device, this information is displayed in real time on a dashboard, helping them to quickly decide on recovery instructions.

[0621] In addition, when a field activity team inputs a question into their smartphone, such as "Where is the nearest evacuation shelter?", the server checks the database and generates the answer, "It's at the community center, a 10-minute walk away," which is displayed on the field activity team's device. This allows necessary information to be provided quickly, allowing field activities to proceed efficiently.

[0622] Example prompts for generative AI models

[0623] 1. "The field team reports that Building A has collapsed and 10 people are injured. I want to analyze and organize this information and display it on my dashboard."

[0624] 2. "In response to a question from the field team, 'Where is the nearest evacuation shelter?', please check the database and generate an appropriate answer."

[0625] These prompts facilitate collaboration between the server and the AI, enabling them to effectively utilize input information from the user.

[0626] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0627] Now, let's explain the processing flow of the system program by dividing it into specific processing steps, and add inputs, outputs, and specific actions to each step.

[0628] ---

[0629] Step 1:

[0630] Input: The user launches the app and enters their ID and password on the login screen.

[0631] Specific operation: The user starts the device, opens the app, and accesses the login screen. They enter their ID and password and click the "Login" button.

[0632] Output: The entered ID and password are sent from the terminal to the server.

[0633] Step 2:

[0634] Input: Login information (ID and password)

[0635] Specific operation: The terminal uses a secure protocol (e.g., TLS / SSL) to encrypt the entered ID and password and send them to the server.

[0636] Output: Login information received by the server

[0637] Step 3:

[0638] Input: Login information received by the server

[0639] Specific operation: The server connects to a database (e.g., MySQL, PostgreSQL), compares the received ID and password with the registered information in the database, generates an authentication result, and returns the result (success or failure) to the terminal.

[0640] Output: The authentication result is generated and returned to the terminal.

[0641] Step 4:

[0642] Input: Authentication result

[0643] Specific operation: The device receives the authentication result, and if successful, allows the user to access the dashboard. A message indicating authentication success is displayed, and the device transitions to the dashboard screen. If unsuccessful, an error message is displayed.

[0644] Output: User accesses dashboard or is returned to login screen again

[0645] Step 5:

[0646] Input: User (on-site activity team) inputs the situation on-site in chat format

[0647] Specific operations: A user of the field operation team enters on-site information using the chat interface on the device and clicks the "Send" button.

[0648] Output: The entered site conditions are sent from the terminal to the server.

[0649] Step 6:

[0650] Input: Site status information

[0651] Specific operation: The server uses AI (natural language processing technology) to analyze the received information and organize it into specific categories (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10."). The organized information is then stored in a database.

[0652] Output: The analyzed and organized information is saved in a database.

[0653] Step 7:

[0654] Input: Organized information

[0655] Specific operation: The server prepares the organized information for display on the dashboard and updates the dashboard.

[0656] Output: The dashboard is updated to show organized information

[0657] Step 8:

[0658] Input: User (on-site activity team) enters the inquiry details

[0659] Specific operation: A user from the field team uses the chat interface on the device to enter a question and press the "send" button.

[0660] Output: The query is sent to the server

[0661] Step 9:

[0662] Input: Inquiry details

[0663] How it works: The server uses AI to analyze the received inquiry and compares it with the database and the latest information. An appropriate answer is generated (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away."). The answer is then sent to the device.

[0664] Output: The generated answer is sent to the device.

[0665] Step 10:

[0666] Input: Answer

[0667] Specific behavior: The device displays the received response in the chat interface within the app.

[0668] Output: The answer is displayed to the user

[0669] Step 11:

[0670] Input: General information updated regularly

[0671] How it works: The server periodically aggregates all information, and AI organizes it visually for easy understanding. The organized information is displayed on a dashboard, and notifications are sent to all user devices when the information is updated.

[0672] Output: Overall information is displayed on the dashboard and notifications are sent when information is updated.

[0673] Step 12:

[0674] Input: Information update notification

[0675] What happens: All user devices receive notifications and access the dashboard.

[0676] Output: User accesses the latest global information

[0677] This will allow you to understand the processing flow of the entire system in concrete detail.

[0678] (Application example 1)

[0679] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0680] There is a problem with insufficient information sharing between the field activity team and the control team during emergencies, making it difficult to efficiently manage and coordinate information in real time. In particular, there is a need for a system that can respond quickly and accurately to reports and inquiries from the field and visually grasp the overall situation.

[0681] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0682] In this invention, the server includes means for receiving information entered by the field activity team and having an AI analyze and organize the information, means for storing the organized information in a database and displaying it on a dashboard, means for the server to periodically aggregate all information and visually display overall information, means for sending notifications to multiple user devices, means for the AI ​​to automatically respond to inquiries from the field activity team, means for making overall information accessible from user devices, means for user authentication, means for reporting emergencies in chat format, means for visually displaying analyzed information, means for using an AI model to compare data and generate responses in an emergency, means for transmitting the generated responses to user devices, and means for updating and displaying information on user devices in real time. This allows information to be shared quickly in real time between the field activity team and the control team, enabling efficient emergency response.

[0683] The "on-site activity team" is a team that actually collects information at the scene of an emergency or disaster and reports on it.

[0684] The "control team" is a central management team that issues instructions based on information provided by the field operation team.

[0685] "AI" is an abbreviation for artificial intelligence, a technology that analyzes data and automatically makes judgments and predictions based on the results.

[0686] A "server" is a computer system that centrally manages information and executes various processes.

[0687] "User devices" are terminals such as smartphones and tablets used by field teams and control teams.

[0688] A "dashboard" is a screen or interface that visually displays information and allows users to grasp the situation in real time.

[0689] A "database" is an information storage system that stores organized information and allows it to be searched and used as needed.

[0690] "Notifications" is a feature that sends information updates and important announcements to user devices.

[0691] "User authentication" is a procedure for verifying that a user is a legitimate user.

[0692] "Chat format" is a text-based communication method that allows for real-time conversation.

[0693] "Analysis" is the process of analyzing information and organizing it into a meaningful form.

[0694] "Visual display" means displaying information in an easy-to-understand manner using graphs, maps, etc.

[0695] "Data matching" is the process of comparing input information with existing data to see if there is a match.

[0696] "Answer generation" is a function that automatically creates an appropriate response based on the results of data matching.

[0697] "Real-time" means that information is processed and notified within a very short time after it is generated.

[0698] This invention is a system that allows field teams and control teams to share information quickly and accurately in an emergency. The system receives information entered by the field team and uses AI to analyze and organize it, allowing the control team to grasp the situation in real time.

[0699] The system mainly consists of a server and a user device.

[0700] Hardware and Software

[0701] Hardware:

[0702] Server (Central Information Management System)

[0703] Smartphones (devices for the field activity team and control team)

[0704] software:

[0705] Flask (Python framework)

[0706] AI module (for data analysis)

[0707] Server Processing

[0708] The server receives information reported by the field teams, which are then analyzed by AI. The analyzed information is organized into a visually easy-to-understand format and stored in a database. It is then displayed in real time on a dashboard. The AI ​​also automatically responds to inquiries from the field teams, generating appropriate answers and sending them to the field teams' user devices.

[0709] User Device Capabilities

[0710] The user devices are smartphones used by the field operation team and the control team. These devices perform user authentication and have the function of reporting emergencies in chat format. The reported information is sent to a server, and the analysis results are displayed on the user device in real time.

[0711] Specific examples

[0712] For example, if a field team reports using their smartphone that "Building B is on fire, 5 injured," the information is analyzed by the server and displayed in organized form as "Location: Building B, Situation: Fire, Number of injured: 5." The control team can view the dashboard and issue instructions quickly based on the latest information. Also, if the field team asks, "Where is the nearest evacuation shelter?" the AI ​​will check the database and answer, "The nearest evacuation shelter is the community center, a 10-minute walk away," and send this information to the field team's device.

[0713] Prompt Sentence Examples

[0714] "As an AI model for a safety management system, please analyze the following information: Building B is on fire, 5 injured."

[0715] In this way, information management and sharing during emergencies can be carried out quickly and accurately, enabling efficient responses.

[0716] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0717] Step 1:

[0718] The user starts up their smartphone device and enters their ID and password on the login screen. The device sends the entered ID and password to the server. The server checks the received user information in a database and returns the authentication result to the device. The input is the user ID and password, and the output is the authentication result. Data processing involves verifying the user information, thereby confirming that the user is legitimate.

[0719] Step 2:

[0720] The user (on-site action team) reports the status of the emergency in chat format. Specifically, they input text information such as "Building B is on fire, 5 injured." The device sends this report to the server. The server then gives the received information to the AI ​​for analysis. The input is the emergency report, and the output is the analyzed specific situation information. Data processing involves text analysis, which organizes the information.

[0721] Step 3:

[0722] The server stores the information analyzed by AI in a database. The organized information is then displayed in real time on a dashboard. The input is the information analyzed by AI, and the output is the information stored in the database and displayed on the dashboard. Data processing involves storing and visualizing the information, making it usable.

[0723] Step 4:

[0724] The local activity team queries the AI ​​for the necessary information. Specifically, they input the question, "Where is the nearest evacuation shelter?" The device sends this query to the server. The server's AI compares the latest information with the database and generates an appropriate answer. The input is the query, and the output is the generated answer. Data processing involves data comparison and answer generation, which provides appropriate information.

[0725] Step 5:

[0726] The server sends the generated response to the terminal of the field team, who then checks the response on their terminal. The input is the response information from the server, and the output is the response displayed on the user's terminal. Data processing involves sending and displaying the information, which allows the field team to obtain the necessary information.

[0727] Step 6:

[0728] The server periodically aggregates all information, and AI organizes it in a visually easy-to-understand format. Specifically, the overall information is displayed in the form of graphs and maps. The input is individual pieces of information data, and the output is overall information that has been visually organized. Data processing involves aggregating and visualizing the information, which allows the overall situation to be grasped.

[0729] Step 7:

[0730] Every time the information is updated, the server sends notifications to all user devices in real time. The input is the new updated information, and the output is the notification to the user device. The data processing is the notification generation and transmission, so that users receive the latest information in real time.

[0731] In this way, by specifically showing how input data is processed at each step and what output is obtained, it becomes easier to understand the flow of the entire system.

[0732] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0733] This invention is a system for smooth information sharing between the control team and the field activity team, and is combined with an emotion engine that recognizes the user's emotions. This system consists of devices used by the control team and the field activity team, and a central server that manages information. The server uses AI and the emotion engine to receive, analyze, organize, and share information as appropriate.

[0734] Program processing flow (natural language explanation)

[0735] 1. User Authentication

[0736] Device: The user launches the app and enters their ID and password on the login screen.

[0737] Terminal: Sends the entered ID and password to the server.

[0738] Server: Checks the received user information against the database and generates an authentication result.

[0739] Server: If authentication is successful, generates an authentication token and sends it to the terminal. If authentication fails, sends an error message to the terminal.

[0740] Terminal: Displays the authentication result (success or failure) to the user.

[0741] 2. Information gathering and interviews

[0742] User (on-site activity team): Enters the situation at the scene in chat format (e.g., "Building A has collapsed, 10 injured.").

[0743] Terminal: Sends user input to the server.

[0744] Server: The received information is analyzed and the AI ​​classifies the content in detail (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10.").

[0745] Server: Stores organized information in a database.

[0746] Server: Organized information is displayed in real time on a dashboard.

[0747] Terminal (control team): Organized information is displayed on a dashboard, allowing the control team to check the latest information.

[0748] 3. Automated response to inquiries

[0749] User (Field Operations Team): Asks the AI ​​for specific information (e.g., "Where is the nearest evacuation shelter?").

[0750] Terminal: Sends the query to the server.

[0751] Server: The AI ​​analyzes the incoming inquiries and compares them with the latest information in the database.

[0752] Server: Generates an appropriate answer (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away.").

[0753] Server: Sends the generated answers to the terminals of the field activity team.

[0754] Terminal (Field Team): Displays the generated answers to the user.

[0755] 4. Organizing and sharing overall information

[0756] Server: All collected information is periodically aggregated and systematically organized by AI.

[0757] Server: View organized overview information on the dashboard.

[0758] Server: Sends real-time notifications to user devices whenever information is updated.

[0759] Devices (Control and Field Operations Teams): Receive notifications and access the dashboard for the latest information.

[0760] Terminals (control team and field operation team): Detailed information can be retrieved from the dashboard as needed to help understand the situation on site and carry out rescue operations.

[0761] 5. Recognizing and responding to user emotions

[0762] User (field team or control team): Expresses emotions (e.g., tension, anger) when interacting with the system through tone of voice and input text.

[0763] Terminal: Sends the user's emotional information to the server.

[0764] Server: The emotion engine analyzes the user's tone of voice and text to identify emotions (e.g., if the voice is trembling, it is recognized as nervousness).

[0765] Server: Generates an appropriate response based on the identified emotion (e.g., a calming message to ease tension).

[0766] Server: Sends the generated response to the user device and displays it to the user.

[0767] Specific examples

[0768] For example, if building A collapses during a disaster, an operator from the local activity team can type "Building A collapsed, 10 injured" into the chat on their smartphone. This information is sent to the server, where it is analyzed by AI and organized as "Location: Building A, Situation: Collapsed, Number of injured: 10." The organized information is displayed on the control team's dashboard, contributing to the prompt issuing of recovery instructions.

[0769] Furthermore, when a field team asks, "Where is the nearest evacuation shelter?", the server checks the database and replies, "It's at the community center, a 10-minute walk away," which is displayed on the field team's terminal. If the operator's voice is trembling, the emotion engine recognizes this as "tension" and automatically generates a response such as, "Please stay calm, the evacuation shelter is nearby."

[0770] This allows for smooth sharing of information between the field activity team and the control team, reducing misunderstandings and duplicate questions, enabling efficient disaster response, and also responding to users' emotions.

[0771] The processing flow will be explained below.

[0772] Specific processing steps of the program

[0773] User Authentication Process

[0774] Step 1:

[0775] Device: The user launches the app and enters their ID and password on the login screen.

[0776] Step 2:

[0777] Terminal: Sends the entered ID and password to the server.

[0778] Step 3:

[0779] Server: Compares the received ID and password with the database and generates an authentication result.

[0780] Step 4:

[0781] Server: If authentication is successful, generates an authentication token and sends it to the terminal. If authentication fails, sends an error message to the terminal.

[0782] Step 5:

[0783] Terminal: Displays the authentication result (success or failure) to the user.

[0784] Information gathering and interviews

[0785] Step 1:

[0786] User (on-site activity team): Enters the situation at the scene in chat format (e.g., "Building A has collapsed, 10 injured.").

[0787] Step 2:

[0788] Terminal: Sends user input to the server.

[0789] Step 3:

[0790] Server: The received information is analyzed and the AI ​​classifies the content in detail (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10.").

[0791] Step 4:

[0792] Server: Stores organized information in a database.

[0793] Step 5:

[0794] Server: Organized information is displayed in real time on a dashboard.

[0795] Step 6:

[0796] Terminal (control team): Organized information is displayed on a dashboard, allowing the control team to check the latest information.

[0797] Automated response to inquiries

[0798] Step 1:

[0799] User (Field Operations Team): Asks the AI ​​for specific information (e.g., "Where is the nearest evacuation shelter?").

[0800] Step 2:

[0801] Terminal: Sends the query to the server.

[0802] Step 3:

[0803] Server: The AI ​​analyzes the incoming inquiries and compares them with the latest information in the database.

[0804] Step 4:

[0805] Server: Generates an appropriate answer (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away.").

[0806] Step 5:

[0807] Server: Sends the generated answers to the terminals of the field activity team.

[0808] Step 6:

[0809] Terminal (Field Team): Displays the generated answers to the user.

[0810] Organizing and sharing overall information

[0811] Step 1:

[0812] Server: All collected information is periodically aggregated and systematically organized by AI.

[0813] Step 2:

[0814] Server: View organized overview information on the dashboard.

[0815] Step 3:

[0816] Server: Sends real-time notifications to user devices whenever information is updated.

[0817] Step 4:

[0818] Devices (Control and Field Operations Teams): Receive notifications and access the dashboard for the latest information.

[0819] Step 5:

[0820] Terminals (control team and field operation team): Detailed information can be retrieved from the dashboard as needed to help understand the situation on site and carry out rescue operations.

[0821] Recognizing and responding to user emotions

[0822] Step 1:

[0823] Users (field or control team): Express emotions (e.g., tension, anger) when interacting with the system through tone of voice and input text.

[0824] Step 2:

[0825] Terminal: Sends the user's emotional information to the server.

[0826] Step 3:

[0827] Server: The emotion engine analyzes the user's tone of voice and text to identify emotions (e.g., if the voice is trembling, it is recognized as nervousness).

[0828] Step 4:

[0829] Server: Generates an appropriate response based on the identified emotion (e.g., a calming message to ease tension).

[0830] Step 5:

[0831] Server: Sends the generated response to the user device and displays it to the user.

[0832] Example 2

[0833] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0834] When coordinating information between the field operation team and the control team, it is difficult to share accurate and appropriate information in real time. In addition, there is a need to recognize the emotions of users working in the field and respond appropriately, but current systems do not adequately fulfill this requirement.

[0835] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0836] In this invention, the server includes means for receiving information entered by the field activity team and analyzing and organizing the information using a machine learning model, means for storing the organized information in a database and displaying it on an information display device, means for a management device to periodically aggregate all information and visually display overall information, means for sending notifications to multiple user terminals, means for a generation engine to automatically respond to inquiries from the field activity team, means for analyzing tone of voice and input text using an emotion engine that recognizes user emotions and generating an appropriate response message, and means for making overall information accessible from user terminals. This enables accurate and appropriate information sharing in real time between the field activity team and the control team, and realizes flexible responses according to user emotions.

[0837] "Field activity teams" are personnel or teams that mainly operate on-site, reporting on the situation and gathering information.

[0838] "Machine learning models" refer to the algorithms and mathematical models used by the server to perform data analysis and pattern recognition.

[0839] "Means of organizing" refers to the function of receiving and analyzing information, classifying and arranging the content in a specific format, and storing and displaying it.

[0840] A "database" is an information system designed to efficiently store and manage organized information and enable quick retrieval when needed.

[0841] "Information display device" refers to equipment or software that provides an interface for visually checking and manipulating collected information.

[0842] "Management device" refers to a computer system that centrally manages the collection, organization, and display of all information.

[0843] "User terminal" refers to a device (e.g., smartphone, tablet, PC) used by the field activity team and control team that can input and output information.

[0844] "Means for sending notifications" refers to a function that allows the server to notify the user terminal of information updates and important events in real time.

[0845] "Generation engine" refers to AI technology that automatically generates appropriate answers to user inquiries.

[0846] An "emotion engine" refers to AI technology that analyzes emotions from the user's tone of voice and input text, and automatically generates a response appropriate to those emotions.

[0847] "Comprehensive information" refers to comprehensive information that integrates and organizes all data collected from field teams and other sources.

[0848] "Visual display means" refers to the function of displaying collected and organized information in the form of graphs, charts, text, etc. in a way that is easy to understand at a glance.

[0849] "Means of making it accessible" refers to the mechanism that provides the authority and functions for a user device to view and operate the entire information.

[0850] This invention is a system for facilitating information sharing between field teams and control teams, and is combined with an emotion engine that recognizes user emotions. This system consists of devices used by the field teams and control teams, and a central server that manages information. The server utilizes machine learning models and the emotion engine to receive, analyze, organize, share appropriate information, and respond to emotions.

[0851] The server is equipped with a database, machine learning model, and emotion engine. PostgreSQL is used as the database, and TensorFlow is used as the machine learning model. The emotion engine uses IBM Watson Tone Analyzer. Additionally, Grafana is used as the information display device, and Firebase Cloud Messaging (FCM) is used for the notification function.

[0852] Users of the field response team interactively input the situation at the scene. This information is sent to the server via their terminal. The server analyzes the received information using a machine learning model and categorizes and organizes it in detail. For example, an input such as "Building A collapsed, 10 injured" is organized as "Location: Building A, Situation: Collapsed, Number of injured: 10." The organized information is stored in a PostgreSQL database.

[0853] The server then displays the organized information in real time on the Grafana dashboard via the management device, allowing the control team to view the latest information on their terminals. At the same time, the server periodically aggregates all the information and provides a means for visually displaying it. Every time this information is updated, the server sends a real-time notification to the user's terminal.

[0854] Additionally, when the field team makes a direct inquiry to the server, such as "Where is the nearest evacuation shelter?", the server automatically generates an appropriate answer using a generative AI model (OpenAI GPT-4 model). The answer automatically generated is "It's at the community center, a 10-minute walk away," and is displayed on the field team's terminal.

[0855] Furthermore, an emotion engine (IBM Watson Tone Analyzer) analyzes emotions expressed by the user while interacting with the system, such as tension or anger, and automatically generates an appropriate response message. If the user is nervous, a message such as "Please stay calm, a shelter is nearby" is generated and sent to the user's device.

[0856] For example, when a disaster occurs, an operator from the field team can type "Building A has collapsed, 10 injured" into the chat on their smartphone. This information is analyzed and organized on the server and displayed on the control team's dashboard. If the field team asks, "Where is the nearest evacuation shelter?", the server will respond with "The community center, a 10-minute walk away." At the same time, if the operator is nervous, a message will be sent saying, "Please stay calm, the evacuation shelter is nearby."

[0857] Examples of prompts are:

[0858] "Building A has collapsed. 10 people are injured. Please tell me the nearest evacuation shelter."

[0859] "I want to see the current status of injured people in real time. How can I access the dashboard?"

[0860] "What kind of message would be effective for a user who is nervous?"

[0861] This enables accurate and appropriate information sharing in real time between the field operation team and the control team, and also enables flexible responses to user emotions.

[0862] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0863] Program processing flow and specific explanation of each processing step

[0864] Step 1: User authentication

[0865] 1. Input: The user enters the ID and password into the terminal.

[0866] 2. Operation: The device hashes the entered ID and password and sends the data to the server via secure communication (HTTPS).

[0867] 3. Server:

[0868] Data processing: The server receives the hashed user information and checks it against the MySQL database.

[0869] Data operation: Executes SQL queries to retrieve user information and generate authentication results.

[0870] 4. Output: If authentication is successful, a JWT (JSON Web Token) is generated and sent to the terminal. If authentication fails, an error message is generated and sent.

[0871] Step 2: Gathering information and conducting interviews

[0872] 1. Input: The user (on-site activity team) inputs the situation on the scene in chat format. Example: "Building A collapsed, 10 injured."

[0873] 2. Operation: The device sends the input information to the server via HTTPS.

[0874] 3. Server:

[0875] Data processing: The text information received by the server is analyzed using TensorFlow's machine learning model and classified in detail.

[0876] Data calculation: Classify the information as "Location: Building A, Situation: Collapsed, Number of injured: 10."

[0877] 4. Output: The classified information is stored in a PostgreSQL database and sent to an information display device (Grafana) in real time for display.

[0878] Step 3: Automate your inquiries

[0879] 1. Input: A user (field operations team) asks for specific information, e.g., "Where is the nearest evacuation shelter?"

[0880] 2. Operation: The device sends the query to the server via HTTPS.

[0881] 3. Server:

[0882] Data processing: The query received by the server is input into the OpenAI GPT-4 model for analysis.

[0883] Data calculations: AI compares databases with the latest information and generates appropriate answers.

[0884] 4. Output: The generated answer is sent to the device and displayed to the user. For example, "The nearest evacuation shelter is the Civic Center, a 10-minute walk away."

[0885] Step 4: Organize and share overall information

[0886] 1. Input: All collected information is collected on the server.

[0887] 2. Operation: The server periodically aggregates this information.

[0888] 3. Server:

[0889] Data processing: Information is systematically organized using deep learning models running on NVIDIA GPUs with CUDA acceleration.

[0890] Data calculation: Visually organize the overall information.

[0891] 4. Output: The organized overall information is displayed on a Grafana dashboard, and notifications are sent to the user's device using Firebase Cloud Messaging (FCM) every time the information is updated.

[0892] Step 5: Recognize and respond to user emotions

[0893] 1. Input: The user expresses their emotions through tone of voice and text while interacting with the system.

[0894] 2. Operation: The device sends voice and text information expressing emotions to the server via WebRTC.

[0895] 3. Server:

[0896] Data processing: The emotion engine (IBM Watson Tone Analyzer) analyzes the received voice and text to identify emotions.

[0897] Data Calculation: Based on the analysis results, an appropriate response message is generated.

[0898] 4. Output: Send the generated message to the terminal and display it to the user. Example: "Please stay calm, shelter is nearby."

[0899] This allows the system to process information quickly and accurately at each stage, enabling real-time information sharing between the field operation team and the control team, while also enabling it to flexibly respond to user emotions.

[0900] (Application example 2)

[0901] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0902] Conventional information sharing systems for on-site work have had issues such as time-consuming information organization and analysis, and difficulty in responding immediately in emergencies. Furthermore, the lack of feedback that takes into account the emotions and mental state of workers makes it difficult to create an efficient and safe work environment. This invention solves these issues by providing rapid and accurate information sharing and feedback that also takes into account the mental state of workers.

[0903] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving information entered by the field activity team and analyzing and organizing the information using AI, means for storing the organized information in a database and displaying it on a dashboard, and means for periodically aggregating information and visually displaying the overall information. This enables rapid organization and sharing of information. In addition, by using means for recognizing user emotions and providing appropriate feedback, a safe and efficient work environment for workers is ensured.

[0904] A "field unit" is a person or team whose role is to gather and report information at a specific site.

[0905] "AI" refers to the intelligent behavior and learning capabilities imitated by computer systems as artificial intelligence.

[0906] A "dashboard" is an interface for visually displaying collected information, allowing users to check the situation in real time.

[0907] "Server" means a central management system that receives, analyzes, stores, and notifies user devices of information.

[0908] "User device" refers to a terminal used by field teams and control teams, including smartphones, tablets, smart glasses, etc.

[0909] "Sending notifications" refers to the act of automatically sending updates or responses from the server to the user device.

[0910] An "emotion engine" is a technology that recognizes emotions by analyzing the user's tone of voice and input text.

[0911] "Feedback" is the act of providing appropriate responses or instructions according to the user's emotions and situation.

[0912] "Analyzing and organizing information" means that the AI ​​processes, classifies, and reconstructs the information entered by the field team into an easy-to-understand format.

[0913] This invention is a system that enables smooth information sharing between field teams and control teams, and combines an emotion engine that recognizes the user's emotions. The field teams are responsible for collecting and reporting information at specific sites, while the control teams are responsible for managing information centrally and issuing instructions.

[0914] Hardware and software used

[0915] Hardware:

[0916] Smart glasses (e.g. Microsoft HoloLens)

[0917] Server (e.g. AWS EC2)

[0918] software:

[0919] AI analysis engine (e.g. TensorFlow)

[0920] Emotion recognition engine (e.g. IBM Watson Tone Analyzer)

[0921] Database (e.g. MySQL)

[0922] Front-end frameworks (e.g., React.js)

[0923] Backend frameworks (e.g., Node.js)

[0924] Explanation of the generated program

[0925] 1. User authentication:

[0926] Terminal: The user turns on the smart glasses and enters their ID and password. The entered information is sent to the server.

[0927] Server: Checks the received user information in the database and generates an authentication result. If authentication is successful, generates a token and sends it to the terminal.

[0928] Terminal: Displays the authentication result to the user.

[0929] 2. Information gathering and interviews:

[0930] User (field operation team): Collects on-site information using the smart glasses' camera and voice input (e.g., "The conveyor belt has stopped").

[0931] Terminal: Sends the entered information to the server.

[0932] Server: The AI ​​analysis engine processes the received information and classifies it in detail.

[0933] Server: Stores organized information in a database and displays it on a dashboard.

[0934] 3. Automated response to inquiries:

[0935] User (Field Team): Asks the AI ​​for specific information via smart glasses (e.g., "Where is the manual for this device?").

[0936] Terminal: Sends the query to the server.

[0937] Server: The AI ​​analysis engine analyzes the received inquiry, compares it with the database and the latest information, and generates an appropriate answer.

[0938] Server: Sends the generated answer to the user's terminal and displays it to the user.

[0939] 4. Emotion Recognition and Feedback:

[0940] User (field team or control team): Expresses emotions through tone of voice and text input.

[0941] Terminal: Sends emotion information to the server.

[0942] Server: The emotion recognition engine analyzes this and identifies the emotion.

[0943] Server: Generates appropriate feedback, sends it to the user's device, and displays it (e.g., "Please stay calm, our support team will assist you").

[0944] Specific examples

[0945] For example, if a problem occurs in a factory where the conveyor belt has stopped, a worker on the field team can use smart glasses to capture the situation with a camera and report the situation by voice, saying, "The conveyor belt has stopped." This information is sent to the server, where the AI ​​analysis engine analyzes it and classifies it as "Location: Conveyor belt, Status: Stopped," and displays it on a dashboard. The control team can then quickly issue instructions by looking at the dashboard. Furthermore, if the worker's trembling voice is identified as "tension" by the emotion recognition engine, feedback such as "Please stay calm and we will contact the technical support team" is automatically returned.

[0946] Example prompt sentence:

[0947] "A worker in a factory uses smart glasses to report a machine problem. Write the steps to resolve this problem. Also, include an action that recognizes the worker's emotions and provides appropriate feedback."

[0948] This allows the field team and the control team to share information quickly and accurately and take immediate action.It also responds to the emotions of workers, providing a safer and more efficient working environment.

[0949] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0950] Step 1: User authentication

[0951] Input: The user turns on the smart glasses and enters their ID and password.

[0952] Terminal operation: The entered ID and password are sent to the server.

[0953] Server operation: The received user information is checked against the database, and an authentication result is generated. If authentication is successful, an authentication token is generated and sent to the terminal.

[0954] Output: The terminal displays the authentication result to the user.

[0955] Step 2: Information gathering and interviews

[0956] Input: The user (field team) takes pictures of the scene with the camera on the smart glasses and inputs information by voice. For example, they report that the conveyor belt has stopped.

[0957] Device operation: Sends the user's voice input and camera images to the server.

[0958] Server operation: The AI ​​analysis engine processes the received information and classifies it in detail. For example, "Location: conveyor belt, status: stopped."

[0959] Output: The server stores the organized information in a database and displays it on a dashboard.

[0960] Step 3: Automate your inquiries

[0961] Input: A user (field team) queries the AI ​​for specific information via smart glasses, for example, "Where is the manual for this device?"

[0962] Terminal operation: Sends the user's inquiry to the server.

[0963] Server operation: The AI ​​analysis engine analyzes the received inquiry, compares it with the database and the latest information, and generates an appropriate answer.

[0964] Output: The server sends the generated answer to the field team's terminal and displays it to the user.

[0965] Step 4: Emotion recognition and feedback

[0966] Input: Users (field or control teams) express emotions through tone of voice or text input.

[0967] Device operation: Sends user's voice and text information to the server.

[0968] Server operation: The emotion recognition engine analyzes this and identifies the user's emotion. For example, it recognizes a trembling voice as "tension."

[0969] Output: The server generates appropriate feedback (e.g., "Please stay calm, we will contact our technical support team") and sends it to the user's device for display.

[0970] Step 5: Organize and share the overall information

[0971] Input: All information aggregated on the server.

[0972] Server operation: The server periodically aggregates all information, and the AI ​​analysis engine systematically organizes it.

[0973] Output: The organized information is visually displayed on a dashboard, and real-time notifications are sent to the user's device whenever the information is updated.

[0974] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0975] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0976] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0977] [Third embodiment]

[0978] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0979] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0980] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0981] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0982] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0983] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0984] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0985] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0986] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0987] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0988] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0989] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0990] The present invention is a system for facilitating information sharing between the control team and the field activity team. The program processing in this system will be explained below.

[0991] The system consists of devices used by the control team and field teams, and a central server that manages information. The server uses AI to receive, analyze, organize, and share appropriate information. The system also sends real-time notifications to user devices, ensuring that users always have access to the latest information.

[0992] Program processing flow (natural language explanation)

[0993] 1. User Authentication

[0994] Device: The user launches the app and enters their ID and password on the login screen.

[0995] Terminal: Sends the entered ID and password to the server.

[0996] Server: Checks the received user information in the database and returns the authentication result to the terminal.

[0997] Terminal: If authentication is successful, grant the user access to the dashboard.

[0998] 2. Information gathering and interviews

[0999] User (on-site action team): Reports the situation on-site in chat format (e.g., "The building has collapsed. 10 people injured.").

[1000] Terminal: Sends user input to the server.

[1001] Server: Receives information, and the AI ​​analyzes and organizes it into detailed items (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10.").

[1002] Server: Stores organized information in a database and displays it on a dashboard.

[1003] Terminal (control team): Organized information is displayed in real time on the dashboard.

[1004] 3. Automated response to inquiries

[1005] User (on-site response team): Asks the AI ​​for the necessary information (e.g., "Where is the nearest evacuation shelter?").

[1006] Terminal: Sends the query to the server.

[1007] Server: The AI ​​compares the database with the latest information and generates an appropriate answer (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away.").

[1008] Server: Sends the generated answers to the terminals of the field activity team.

[1009] Terminal (Field Team): The answer is displayed to the user.

[1010] 4. Organizing and sharing overall information

[1011] Server: All information is periodically aggregated and AI organizes it in a visually easy-to-understand manner.

[1012] Server: View overall information on the dashboard.

[1013] Server: Sends notifications to all user devices when information is updated.

[1014] Terminals (Control Team and Field Operations Team): View the dashboard and access real-time updated information.

[1015] Specific examples

[1016] For example, if building A collapses during a disaster, an operator from the local activity team can type "Building A collapsed, 10 injured" into chat on their smartphone. This information is sent to the server, where it is analyzed by AI and organized as "Location: Building A, Situation: Collapsed, Number of injured: 10." The organized information is displayed on the control team's dashboard, contributing to the rapid decision-making of restoration instructions.

[1017] Additionally, when a local activity team asks, "Where is the nearest evacuation shelter?" the server checks the database and replies, "It's at the community center, a 10-minute walk away," which is displayed on the local activity team's terminal.

[1018] This allows information to be shared smoothly between the field activity team and the control team, reducing misunderstandings and duplicate questions, and enabling efficient disaster response.

[1019] The processing flow will be explained below.

[1020] Specific processing steps of the program

[1021] User Authentication Process

[1022] Step 1:

[1023] Device: The user launches the app and enters their ID and password on the login screen.

[1024] Step 2:

[1025] Terminal: Sends the entered ID and password to the server.

[1026] Step 3:

[1027] Server: Compares the received ID and password with the database and generates an authentication result.

[1028] Step 4:

[1029] Server: If authentication is successful, generates an authentication token and sends it to the terminal. If authentication fails, sends an error message to the terminal.

[1030] Step 5:

[1031] Terminal: Displays the authentication result (success or failure) to the user.

[1032] Information gathering and interviews

[1033] Step 1:

[1034] User (on-site activity team): Enters the situation at the scene in chat format (e.g., "Building A has collapsed, 10 injured.").

[1035] Step 2:

[1036] Terminal: Sends user input to the server.

[1037] Step 3:

[1038] Server: The received information is analyzed and the AI ​​classifies the content in detail (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10.").

[1039] Step 4:

[1040] Server: Stores organized information in a database.

[1041] Step 5:

[1042] Server: Organized information is displayed in real time on a dashboard.

[1043] Step 6:

[1044] Terminal (control team): Organized information is displayed on a dashboard, allowing the control team to check the latest information.

[1045] Automated response to inquiries

[1046] Step 1:

[1047] User (Field Operations Team): Asks the AI ​​for specific information (e.g., "Where is the nearest evacuation shelter?").

[1048] Step 2:

[1049] Terminal: Sends the query to the server.

[1050] Step 3:

[1051] Server: The AI ​​analyzes the incoming inquiries and compares them with the latest information in the database.

[1052] Step 4:

[1053] Server: Generates an appropriate answer (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away.").

[1054] Step 5:

[1055] Server: Sends the generated answers to the terminals of the field activity team.

[1056] Step 6:

[1057] Terminal (Field Team): Displays the generated answers to the user.

[1058] Organizing and sharing overall information

[1059] Step 1:

[1060] Server: All collected information is periodically aggregated and systematically organized by AI.

[1061] Step 2:

[1062] Server: View organized overview information on the dashboard.

[1063] Step 3:

[1064] Server: Sends real-time notifications to user devices whenever information is updated.

[1065] Step 4:

[1066] Devices (Control and Field Operations Teams): Receive notifications and access the dashboard for the latest information.

[1067] Step 5:

[1068] Terminals (control team and field operation team): Detailed information can be retrieved from the dashboard as needed to help understand the situation on site and carry out rescue operations.

[1069] Example 1

[1070] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1071] There is a problem that information sharing between the field operation team and the control team is not smooth, and it takes a long time to share and analyze information. In addition, it is not possible to respond appropriately to questions and information gathering on the ground in real time, making it difficult to respond to disasters efficiently. Furthermore, there is a lack of visual organization of information and real-time notification of overall information, which makes it difficult for the field and command teams to act quickly.

[1072] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1073] In this invention, the server includes means for receiving information entered by a user and having AI analyze and organize the information, means for storing the organized information in a database and displaying it on a display means, means for the server to periodically aggregate all information and visually display the overall information, means for sending notifications to multiple user devices, means for AI to automatically respond to user inquiries, means for making the overall information accessible from user devices, and means for receiving user authentication information and returning the authentication result to the terminal. This allows information to be shared smoothly between the field activity team and the control team, enabling real-time information provision and response, and achieving efficient disaster response.

[1074] ---

[1075] "User" means a person who uses the system to enter or access information.

[1076] "Server" means a central management system for receiving, analyzing, organizing, storing, displaying, and sending notifications about information.

[1077] "Terminal" refers to a device used by a user, an electronic device that communicates with a server through an application.

[1078] "Information" refers to data such as on-site conditions and inquiries entered by the field activity team and control team.

[1079] "AI" stands for artificial intelligence, and refers to technology that analyzes incoming information, organizes it appropriately, and generates answers.

[1080] A "database" refers to a system for storing organized information and retrieving it as needed.

[1081] "Display means" refers to an interface for visually presenting organized information to a user.

[1082] "Aggregation" refers to the process of bringing together distributed information.

[1083] "Visually displaying" refers to presenting information in a format such as a graph or chart in an easy-to-read format to the user.

[1084] "Notification" refers to a message that informs a user device that new information exists.

[1085] An "inquiry" refers to an action in which a user requests information from a system.

[1086] "Authentication" refers to the process of verifying a user's ID and password and confirming that the user is a legitimate user.

[1087] ---

[1088] ---

[1089] The present invention is a system for facilitating information sharing between the control team and the field activity team. This system consists of terminals used by the control team and the field activity team, and a server that centrally manages information. Below, we will explain in detail how each component works.

[1090] First, a user accesses the system using a terminal. At this time, the user enters their ID and password on the login screen and sends the authentication information to the server. The server compares the received information with the database and returns the authentication result to the terminal. If authentication is successful, the user is allowed to access the dashboard.

[1091] Next, users from the field operation team use their devices to report on the situation at the scene. The users input information in chat format and send it to the server. The server uses AI to analyze the received information and organizes the results of the analysis. For example, if a report is made such as "Building collapsed. 10 injured," the AI ​​will organize this into detailed items such as "Location: Building A, Situation: Collapsed, Number of injured: 10." This organized information is stored in a database and displayed in real time on a dashboard.

[1092] Users can also inquire about the information they need on-site. For example, by asking the AI ​​a question such as "Where is the nearest evacuation shelter?", the AI ​​sends the question to the server. The server then uses the AI ​​to compare the latest information with the database and generate an appropriate answer. The generated answer is sent to the device of the on-site team and displayed to the user.

[1093] The server periodically aggregates all information, and AI organizes it visually for easy understanding. The organized overall information is displayed on a dashboard, and the server sends notifications to all user devices when the information is updated. This allows control teams and field teams to access the latest information in real time and respond quickly.

[1094] Specific examples

[1095] For example, if Building A collapses during a disaster, an operator from the field operation team can enter "Building A collapsed, 10 injured" into the chat app on their smartphone. This information is sent to the server, where the AI ​​analyzes and organizes it as "Location: Building A, Situation: Collapsed, Number of injured: 10," and stores it in a database. On the control team's device, this information is displayed in real time on a dashboard, helping them to quickly decide on recovery instructions.

[1096] In addition, when a field activity team inputs a question into their smartphone, such as "Where is the nearest evacuation shelter?", the server checks the database and generates the answer, "It's at the community center, a 10-minute walk away," which is displayed on the field activity team's device. This allows necessary information to be provided quickly, allowing field activities to proceed efficiently.

[1097] Example prompts for generative AI models

[1098] 1. "The field team reports that Building A has collapsed and 10 people are injured. I want to analyze and organize this information and display it on my dashboard."

[1099] 2. "In response to a question from the field team, 'Where is the nearest evacuation shelter?', please check the database and generate an appropriate answer."

[1100] These prompts facilitate collaboration between the server and the AI, enabling them to effectively utilize input information from the user.

[1101] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1102] Now, let's explain the processing flow of the system program by dividing it into specific processing steps, and add inputs, outputs, and specific actions to each step.

[1103] ---

[1104] Step 1:

[1105] Input: The user launches the app and enters their ID and password on the login screen.

[1106] Specific operation: The user starts the device, opens the app, and accesses the login screen. They enter their ID and password and click the "Login" button.

[1107] Output: The entered ID and password are sent from the terminal to the server.

[1108] Step 2:

[1109] Input: Login information (ID and password)

[1110] Specific operation: The terminal uses a secure protocol (e.g., TLS / SSL) to encrypt the entered ID and password and send them to the server.

[1111] Output: Login information received by the server

[1112] Step 3:

[1113] Input: Login information received by the server

[1114] Specific operation: The server connects to a database (e.g., MySQL, PostgreSQL), compares the received ID and password with the registered information in the database, generates an authentication result, and returns the result (success or failure) to the terminal.

[1115] Output: The authentication result is generated and returned to the terminal.

[1116] Step 4:

[1117] Input: Authentication result

[1118] Specific operation: The device receives the authentication result, and if successful, allows the user to access the dashboard. A message indicating authentication success is displayed, and the device transitions to the dashboard screen. If unsuccessful, an error message is displayed.

[1119] Output: User accesses dashboard or is returned to login screen again

[1120] Step 5:

[1121] Input: User (on-site activity team) inputs the situation on-site in chat format

[1122] Specific operations: A user of the field operation team enters on-site information using the chat interface on the device and clicks the "Send" button.

[1123] Output: The entered site conditions are sent from the terminal to the server.

[1124] Step 6:

[1125] Input: Site status information

[1126] Specific operation: The server uses AI (natural language processing technology) to analyze the received information and organize it into specific categories (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10."). The organized information is then stored in a database.

[1127] Output: The analyzed and organized information is saved in a database.

[1128] Step 7:

[1129] Input: Organized information

[1130] Specific operation: The server prepares the organized information for display on the dashboard and updates the dashboard.

[1131] Output: The dashboard is updated to show organized information

[1132] Step 8:

[1133] Input: User (on-site activity team) enters the inquiry details

[1134] Specific operation: A user from the field team uses the chat interface on the device to enter a question and press the "send" button.

[1135] Output: The query is sent to the server

[1136] Step 9:

[1137] Input: Inquiry details

[1138] How it works: The server uses AI to analyze the received inquiry and compares it with the database and the latest information. An appropriate answer is generated (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away."). The answer is then sent to the device.

[1139] Output: The generated answer is sent to the device.

[1140] Step 10:

[1141] Input: Answer

[1142] Specific behavior: The device displays the received response in the chat interface within the app.

[1143] Output: The answer is displayed to the user

[1144] Step 11:

[1145] Input: General information updated regularly

[1146] How it works: The server periodically aggregates all information, and AI organizes it visually for easy understanding. The organized information is displayed on a dashboard, and notifications are sent to all user devices when the information is updated.

[1147] Output: Overall information is displayed on the dashboard and notifications are sent when information is updated.

[1148] Step 12:

[1149] Input: Information update notification

[1150] What happens: All user devices receive notifications and access the dashboard.

[1151] Output: User accesses the latest global information

[1152] This will allow you to understand the processing flow of the entire system in concrete detail.

[1153] (Application example 1)

[1154] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1155] There is a problem with insufficient information sharing between the field activity team and the control team during emergencies, making it difficult to efficiently manage and coordinate information in real time. In particular, there is a need for a system that can respond quickly and accurately to reports and inquiries from the field and visually grasp the overall situation.

[1156] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1157] In this invention, the server includes means for receiving information entered by the field activity team and having an AI analyze and organize the information, means for storing the organized information in a database and displaying it on a dashboard, means for the server to periodically aggregate all information and visually display overall information, means for sending notifications to multiple user devices, means for the AI ​​to automatically respond to inquiries from the field activity team, means for making overall information accessible from user devices, means for user authentication, means for reporting emergencies in chat format, means for visually displaying analyzed information, means for using an AI model to compare data and generate responses in an emergency, means for transmitting the generated responses to user devices, and means for updating and displaying information on user devices in real time. This allows information to be shared quickly in real time between the field activity team and the control team, enabling efficient emergency response.

[1158] The "on-site activity team" is a team that actually collects information at the scene of an emergency or disaster and reports on it.

[1159] The "control team" is a central management team that issues instructions based on information provided by the field operation team.

[1160] "AI" is an abbreviation for artificial intelligence, a technology that analyzes data and automatically makes judgments and predictions based on the results.

[1161] A "server" is a computer system that centrally manages information and executes various processes.

[1162] "User devices" are terminals such as smartphones and tablets used by field teams and control teams.

[1163] A "dashboard" is a screen or interface that visually displays information and allows users to grasp the situation in real time.

[1164] A "database" is an information storage system that stores organized information and allows it to be searched and used as needed.

[1165] "Notifications" is a feature that sends information updates and important announcements to user devices.

[1166] "User authentication" is a procedure for verifying that a user is a legitimate user.

[1167] "Chat format" is a text-based communication method that allows for real-time conversation.

[1168] "Analysis" is the process of analyzing information and organizing it into a meaningful form.

[1169] "Visual display" means displaying information in an easy-to-understand manner using graphs, maps, etc.

[1170] "Data matching" is the process of comparing input information with existing data to see if there is a match.

[1171] "Answer generation" is a function that automatically creates an appropriate response based on the results of data matching.

[1172] "Real-time" means that information is processed and notified within a very short time after it is generated.

[1173] This invention is a system that allows field teams and control teams to share information quickly and accurately in an emergency. The system receives information entered by the field team and uses AI to analyze and organize it, allowing the control team to grasp the situation in real time.

[1174] The system mainly consists of a server and a user device.

[1175] Hardware and Software

[1176] Hardware:

[1177] Server (Central Information Management System)

[1178] Smartphones (devices for the field activity team and control team)

[1179] software:

[1180] Flask (Python framework)

[1181] AI module (for data analysis)

[1182] Server Processing

[1183] The server receives information reported by the field teams, which are then analyzed by AI. The analyzed information is organized into a visually easy-to-understand format and stored in a database. It is then displayed in real time on a dashboard. The AI ​​also automatically responds to inquiries from the field teams, generating appropriate answers and sending them to the field teams' user devices.

[1184] User Device Capabilities

[1185] The user devices are smartphones used by the field operation team and the control team. These devices perform user authentication and have the function of reporting emergencies in chat format. The reported information is sent to a server, and the analysis results are displayed on the user device in real time.

[1186] Specific examples

[1187] For example, if a field team reports using their smartphone that "Building B is on fire, 5 injured," the information is analyzed by the server and displayed in organized form as "Location: Building B, Situation: Fire, Number of injured: 5." The control team can view the dashboard and issue instructions quickly based on the latest information. Also, if the field team asks, "Where is the nearest evacuation shelter?" the AI ​​will check the database and answer, "The nearest evacuation shelter is the community center, a 10-minute walk away," and send this information to the field team's device.

[1188] Prompt Sentence Examples

[1189] "As an AI model for a safety management system, please analyze the following information: Building B is on fire, 5 injured."

[1190] In this way, information management and sharing during emergencies can be carried out quickly and accurately, enabling efficient responses.

[1191] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1192] Step 1:

[1193] The user starts up their smartphone device and enters their ID and password on the login screen. The device sends the entered ID and password to the server. The server checks the received user information in a database and returns the authentication result to the device. The input is the user ID and password, and the output is the authentication result. Data processing involves verifying the user information, thereby confirming that the user is legitimate.

[1194] Step 2:

[1195] The user (on-site action team) reports the status of the emergency in chat format. Specifically, they input text information such as "Building B is on fire, 5 injured." The device sends this report to the server. The server then gives the received information to the AI ​​for analysis. The input is the emergency report, and the output is the analyzed specific situation information. Data processing involves text analysis, which organizes the information.

[1196] Step 3:

[1197] The server stores the information analyzed by AI in a database. The organized information is then displayed in real time on a dashboard. The input is the information analyzed by AI, and the output is the information stored in the database and displayed on the dashboard. Data processing involves storing and visualizing the information, making it usable.

[1198] Step 4:

[1199] The local activity team queries the AI ​​for the necessary information. Specifically, they input the question, "Where is the nearest evacuation shelter?" The device sends this query to the server. The server's AI compares the latest information with the database and generates an appropriate answer. The input is the query, and the output is the generated answer. Data processing involves data comparison and answer generation, which provides appropriate information.

[1200] Step 5:

[1201] The server sends the generated response to the terminal of the field team, who then checks the response on their terminal. The input is the response information from the server, and the output is the response displayed on the user's terminal. Data processing involves sending and displaying the information, which allows the field team to obtain the necessary information.

[1202] Step 6:

[1203] The server periodically aggregates all information, and AI organizes it in a visually easy-to-understand format. Specifically, the overall information is displayed in the form of graphs and maps. The input is individual pieces of information data, and the output is overall information that has been visually organized. Data processing involves aggregating and visualizing the information, which allows the overall situation to be grasped.

[1204] Step 7:

[1205] Every time the information is updated, the server sends notifications to all user devices in real time. The input is the new updated information, and the output is the notification to the user device. The data processing is the notification generation and transmission, so that users receive the latest information in real time.

[1206] In this way, by specifically showing how input data is processed at each step and what output is obtained, it becomes easier to understand the flow of the entire system.

[1207] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1208] This invention is a system for smooth information sharing between the control team and the field activity team, and is combined with an emotion engine that recognizes the user's emotions. This system consists of devices used by the control team and the field activity team, and a central server that manages information. The server uses AI and the emotion engine to receive, analyze, organize, and share information as appropriate.

[1209] Program processing flow (natural language explanation)

[1210] 1. User Authentication

[1211] Device: The user launches the app and enters their ID and password on the login screen.

[1212] Terminal: Sends the entered ID and password to the server.

[1213] Server: Checks the received user information against the database and generates an authentication result.

[1214] Server: If authentication is successful, generates an authentication token and sends it to the terminal. If authentication fails, sends an error message to the terminal.

[1215] Terminal: Displays the authentication result (success or failure) to the user.

[1216] 2. Information gathering and interviews

[1217] User (on-site activity team): Enters the situation at the scene in chat format (e.g., "Building A has collapsed, 10 injured.").

[1218] Terminal: Sends user input to the server.

[1219] Server: The received information is analyzed and the AI ​​classifies the content in detail (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10.").

[1220] Server: Stores organized information in a database.

[1221] Server: Organized information is displayed in real time on a dashboard.

[1222] Terminal (control team): Organized information is displayed on a dashboard, allowing the control team to check the latest information.

[1223] 3. Automated response to inquiries

[1224] User (Field Operations Team): Asks the AI ​​for specific information (e.g., "Where is the nearest evacuation shelter?").

[1225] Terminal: Sends the query to the server.

[1226] Server: The AI ​​analyzes the incoming inquiries and compares them with the latest information in the database.

[1227] Server: Generates an appropriate answer (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away.").

[1228] Server: Sends the generated answers to the terminals of the field activity team.

[1229] Terminal (Field Team): Displays the generated answers to the user.

[1230] 4. Organizing and sharing overall information

[1231] Server: All collected information is periodically aggregated and systematically organized by AI.

[1232] Server: View organized overview information on the dashboard.

[1233] Server: Sends real-time notifications to user devices whenever information is updated.

[1234] Devices (Control and Field Operations Teams): Receive notifications and access the dashboard for the latest information.

[1235] Terminals (control team and field operation team): Detailed information can be retrieved from the dashboard as needed to help understand the situation on site and carry out rescue operations.

[1236] 5. Recognizing and responding to user emotions

[1237] User (field team or control team): Expresses emotions (e.g., tension, anger) when interacting with the system through tone of voice and input text.

[1238] Terminal: Sends the user's emotional information to the server.

[1239] Server: The emotion engine analyzes the user's tone of voice and text to identify emotions (e.g., if the voice is trembling, it is recognized as nervousness).

[1240] Server: Generates an appropriate response based on the identified emotion (e.g., a calming message to ease tension).

[1241] Server: Sends the generated response to the user device and displays it to the user.

[1242] Specific examples

[1243] For example, if building A collapses during a disaster, an operator from the local activity team can type "Building A collapsed, 10 injured" into the chat on their smartphone. This information is sent to the server, where it is analyzed by AI and organized as "Location: Building A, Situation: Collapsed, Number of injured: 10." The organized information is displayed on the control team's dashboard, contributing to the prompt issuing of recovery instructions.

[1244] Furthermore, when a field team asks, "Where is the nearest evacuation shelter?", the server checks the database and replies, "It's at the community center, a 10-minute walk away," which is displayed on the field team's terminal. If the operator's voice is trembling, the emotion engine recognizes this as "tension" and automatically generates a response such as, "Please stay calm, the evacuation shelter is nearby."

[1245] This allows for smooth sharing of information between the field activity team and the control team, reducing misunderstandings and duplicate questions, enabling efficient disaster response, and also responding to users' emotions.

[1246] The processing flow will be explained below.

[1247] Specific processing steps of the program

[1248] User Authentication Process

[1249] Step 1:

[1250] Device: The user launches the app and enters their ID and password on the login screen.

[1251] Step 2:

[1252] Terminal: Sends the entered ID and password to the server.

[1253] Step 3:

[1254] Server: Compares the received ID and password with the database and generates an authentication result.

[1255] Step 4:

[1256] Server: If authentication is successful, generates an authentication token and sends it to the terminal. If authentication fails, sends an error message to the terminal.

[1257] Step 5:

[1258] Terminal: Displays the authentication result (success or failure) to the user.

[1259] Information gathering and interviews

[1260] Step 1:

[1261] User (on-site activity team): Enters the situation at the scene in chat format (e.g., "Building A has collapsed, 10 injured.").

[1262] Step 2:

[1263] Terminal: Sends user input to the server.

[1264] Step 3:

[1265] Server: The received information is analyzed and the AI ​​classifies the content in detail (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10.").

[1266] Step 4:

[1267] Server: Stores organized information in a database.

[1268] Step 5:

[1269] Server: Organized information is displayed in real time on a dashboard.

[1270] Step 6:

[1271] Terminal (control team): Organized information is displayed on a dashboard, allowing the control team to check the latest information.

[1272] Automated response to inquiries

[1273] Step 1:

[1274] User (Field Operations Team): Asks the AI ​​for specific information (e.g., "Where is the nearest evacuation shelter?").

[1275] Step 2:

[1276] Terminal: Sends the query to the server.

[1277] Step 3:

[1278] Server: The AI ​​analyzes the incoming inquiries and compares them with the latest information in the database.

[1279] Step 4:

[1280] Server: Generates an appropriate answer (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away.").

[1281] Step 5:

[1282] Server: Sends the generated answers to the terminals of the field activity team.

[1283] Step 6:

[1284] Terminal (Field Team): Displays the generated answers to the user.

[1285] Organizing and sharing overall information

[1286] Step 1:

[1287] Server: All collected information is periodically aggregated and systematically organized by AI.

[1288] Step 2:

[1289] Server: View organized overview information on the dashboard.

[1290] Step 3:

[1291] Server: Sends real-time notifications to user devices whenever information is updated.

[1292] Step 4:

[1293] Devices (Control and Field Operations Teams): Receive notifications and access the dashboard for the latest information.

[1294] Step 5:

[1295] Terminals (control team and field operation team): Detailed information can be retrieved from the dashboard as needed to help understand the situation on site and carry out rescue operations.

[1296] Recognizing and responding to user emotions

[1297] Step 1:

[1298] Users (field or control team): Express emotions (e.g., tension, anger) when interacting with the system through tone of voice and input text.

[1299] Step 2:

[1300] Terminal: Sends the user's emotional information to the server.

[1301] Step 3:

[1302] Server: The emotion engine analyzes the user's tone of voice and text to identify emotions (e.g., if the voice is trembling, it is recognized as nervousness).

[1303] Step 4:

[1304] Server: Generates an appropriate response based on the identified emotion (e.g., a calming message to ease tension).

[1305] Step 5:

[1306] Server: Sends the generated response to the user device and displays it to the user.

[1307] Example 2

[1308] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1309] When coordinating information between the field operation team and the control team, it is difficult to share accurate and appropriate information in real time. In addition, there is a need to recognize the emotions of users working in the field and respond appropriately, but current systems do not adequately fulfill this requirement.

[1310] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1311] In this invention, the server includes means for receiving information entered by the field activity team and analyzing and organizing the information using a machine learning model, means for storing the organized information in a database and displaying it on an information display device, means for a management device to periodically aggregate all information and visually display overall information, means for sending notifications to multiple user terminals, means for a generation engine to automatically respond to inquiries from the field activity team, means for analyzing tone of voice and input text using an emotion engine that recognizes user emotions and generating an appropriate response message, and means for making overall information accessible from user terminals. This enables accurate and appropriate information sharing in real time between the field activity team and the control team, and realizes flexible responses according to user emotions.

[1312] "Field activity teams" are personnel or teams that mainly operate on-site, reporting on the situation and gathering information.

[1313] "Machine learning models" refer to the algorithms and mathematical models used by the server to perform data analysis and pattern recognition.

[1314] "Means of organizing" refers to the function of receiving and analyzing information, classifying and arranging the content in a specific format, and storing and displaying it.

[1315] A "database" is an information system designed to efficiently store and manage organized information and enable quick retrieval when needed.

[1316] "Information display device" refers to equipment or software that provides an interface for visually checking and manipulating collected information.

[1317] "Management device" refers to a computer system that centrally manages the collection, organization, and display of all information.

[1318] "User terminal" refers to a device (e.g., smartphone, tablet, PC) used by the field activity team and control team that can input and output information.

[1319] "Means for sending notifications" refers to a function that allows the server to notify the user terminal of information updates and important events in real time.

[1320] "Generation engine" refers to AI technology that automatically generates appropriate answers to user inquiries.

[1321] An "emotion engine" refers to AI technology that analyzes emotions from the user's tone of voice and input text, and automatically generates a response appropriate to those emotions.

[1322] "Comprehensive information" refers to comprehensive information that integrates and organizes all data collected from field teams and other sources.

[1323] "Visual display means" refers to the function of displaying collected and organized information in the form of graphs, charts, text, etc. in a way that is easy to understand at a glance.

[1324] "Means of making it accessible" refers to the mechanism that provides the authority and functions for a user device to view and operate the entire information.

[1325] This invention is a system for facilitating information sharing between field teams and control teams, and is combined with an emotion engine that recognizes user emotions. This system consists of devices used by the field teams and control teams, and a central server that manages information. The server utilizes machine learning models and the emotion engine to receive, analyze, organize, share appropriate information, and respond to emotions.

[1326] The server is equipped with a database, machine learning model, and emotion engine. PostgreSQL is used as the database, and TensorFlow is used as the machine learning model. The emotion engine uses IBM Watson Tone Analyzer. Additionally, Grafana is used as the information display device, and Firebase Cloud Messaging (FCM) is used for the notification function.

[1327] Users of the field response team interactively input the situation at the scene. This information is sent to the server via their terminal. The server analyzes the received information using a machine learning model and categorizes and organizes it in detail. For example, an input such as "Building A collapsed, 10 injured" is organized as "Location: Building A, Situation: Collapsed, Number of injured: 10." The organized information is stored in a PostgreSQL database.

[1328] The server then displays the organized information in real time on the Grafana dashboard via the management device, allowing the control team to view the latest information on their terminals. At the same time, the server periodically aggregates all the information and provides a means for visually displaying it. Every time this information is updated, the server sends a real-time notification to the user's terminal.

[1329] Additionally, when the field team makes a direct inquiry to the server, such as "Where is the nearest evacuation shelter?", the server automatically generates an appropriate answer using a generative AI model (OpenAI GPT-4 model). The answer automatically generated is "It's at the community center, a 10-minute walk away," and is displayed on the field team's terminal.

[1330] Furthermore, an emotion engine (IBM Watson Tone Analyzer) analyzes emotions expressed by the user while interacting with the system, such as tension or anger, and automatically generates an appropriate response message. If the user is nervous, a message such as "Please stay calm, a shelter is nearby" is generated and sent to the user's device.

[1331] For example, when a disaster occurs, an operator from the field team can type "Building A has collapsed, 10 injured" into the chat on their smartphone. This information is analyzed and organized on the server and displayed on the control team's dashboard. If the field team asks, "Where is the nearest evacuation shelter?", the server will respond with "The community center, a 10-minute walk away." At the same time, if the operator is nervous, a message will be sent saying, "Please stay calm, the evacuation shelter is nearby."

[1332] Examples of prompts are:

[1333] "Building A has collapsed. 10 people are injured. Please tell me the nearest evacuation shelter."

[1334] "I want to see the current status of injured people in real time. How can I access the dashboard?"

[1335] "What kind of message would be effective for a user who is nervous?"

[1336] This enables accurate and appropriate information sharing in real time between the field operation team and the control team, and also enables flexible responses to user emotions.

[1337] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1338] Program processing flow and specific explanation of each processing step

[1339] Step 1: User authentication

[1340] 1. Input: The user enters the ID and password into the terminal.

[1341] 2. Operation: The device hashes the entered ID and password and sends the data to the server via secure communication (HTTPS).

[1342] 3. Server:

[1343] Data processing: The server receives the hashed user information and checks it against the MySQL database.

[1344] Data operation: Executes SQL queries to retrieve user information and generate authentication results.

[1345] 4. Output: If authentication is successful, a JWT (JSON Web Token) is generated and sent to the terminal. If authentication fails, an error message is generated and sent.

[1346] Step 2: Gathering information and conducting interviews

[1347] 1. Input: The user (on-site activity team) inputs the situation on the scene in chat format. Example: "Building A collapsed, 10 injured."

[1348] 2. Operation: The device sends the input information to the server via HTTPS.

[1349] 3. Server:

[1350] Data processing: The text information received by the server is analyzed using TensorFlow's machine learning model and classified in detail.

[1351] Data calculation: Classify the information as "Location: Building A, Situation: Collapsed, Number of injured: 10."

[1352] 4. Output: The classified information is stored in a PostgreSQL database and sent to an information display device (Grafana) in real time for display.

[1353] Step 3: Automate your inquiries

[1354] 1. Input: A user (field operations team) asks for specific information, e.g., "Where is the nearest evacuation shelter?"

[1355] 2. Operation: The device sends the query to the server via HTTPS.

[1356] 3. Server:

[1357] Data processing: The query received by the server is input into the OpenAI GPT-4 model for analysis.

[1358] Data calculations: AI compares databases with the latest information and generates appropriate answers.

[1359] 4. Output: The generated answer is sent to the device and displayed to the user. For example, "The nearest evacuation shelter is the Civic Center, a 10-minute walk away."

[1360] Step 4: Organize and share overall information

[1361] 1. Input: All collected information is collected on the server.

[1362] 2. Operation: The server periodically aggregates this information.

[1363] 3. Server:

[1364] Data processing: Information is systematically organized using deep learning models running on NVIDIA GPUs with CUDA acceleration.

[1365] Data calculation: Visually organize the overall information.

[1366] 4. Output: The organized overall information is displayed on a Grafana dashboard, and notifications are sent to the user's device using Firebase Cloud Messaging (FCM) every time the information is updated.

[1367] Step 5: Recognize and respond to user emotions

[1368] 1. Input: The user expresses their emotions through tone of voice and text while interacting with the system.

[1369] 2. Operation: The device sends voice and text information expressing emotions to the server via WebRTC.

[1370] 3. Server:

[1371] Data processing: The emotion engine (IBM Watson Tone Analyzer) analyzes the received voice and text to identify emotions.

[1372] Data Calculation: Based on the analysis results, an appropriate response message is generated.

[1373] 4. Output: Send the generated message to the terminal and display it to the user. Example: "Please stay calm, shelter is nearby."

[1374] This allows the system to process information quickly and accurately at each stage, enabling real-time information sharing between the field operation team and the control team, while also enabling it to flexibly respond to user emotions.

[1375] (Application example 2)

[1376] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1377] Conventional information sharing systems for on-site work have had issues such as time-consuming information organization and analysis, and difficulty in responding immediately in emergencies. Furthermore, the lack of feedback that takes into account the emotions and mental state of workers makes it difficult to create an efficient and safe work environment. This invention solves these issues by providing rapid and accurate information sharing and feedback that also takes into account the mental state of workers.

[1378] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving information entered by the field activity team and analyzing and organizing the information using AI, means for storing the organized information in a database and displaying it on a dashboard, and means for periodically aggregating information and visually displaying the overall information. This enables rapid organization and sharing of information. In addition, by using means for recognizing user emotions and providing appropriate feedback, a safe and efficient work environment for workers is ensured.

[1379] A "field unit" is a person or team whose role is to gather and report information at a specific site.

[1380] "AI" refers to the intelligent behavior and learning capabilities imitated by computer systems as artificial intelligence.

[1381] A "dashboard" is an interface for visually displaying collected information, allowing users to check the situation in real time.

[1382] "Server" means a central management system that receives, analyzes, stores, and notifies user devices of information.

[1383] "User device" refers to a terminal used by field teams and control teams, including smartphones, tablets, smart glasses, etc.

[1384] "Sending notifications" refers to the act of automatically sending updates or responses from the server to the user device.

[1385] An "emotion engine" is a technology that recognizes emotions by analyzing the user's tone of voice and input text.

[1386] "Feedback" is the act of providing appropriate responses or instructions according to the user's emotions and situation.

[1387] "Analyzing and organizing information" means that the AI ​​processes, classifies, and reconstructs the information entered by the field team into an easy-to-understand format.

[1388] This invention is a system that enables smooth information sharing between field teams and control teams, and combines an emotion engine that recognizes the user's emotions. The field teams are responsible for collecting and reporting information at specific sites, while the control teams are responsible for managing information centrally and issuing instructions.

[1389] Hardware and software used

[1390] Hardware:

[1391] Smart glasses (e.g. Microsoft HoloLens)

[1392] Server (e.g. AWS EC2)

[1393] software:

[1394] AI analysis engine (e.g. TensorFlow)

[1395] Emotion recognition engine (e.g. IBM Watson Tone Analyzer)

[1396] Database (e.g. MySQL)

[1397] Front-end frameworks (e.g., React.js)

[1398] Backend frameworks (e.g., Node.js)

[1399] Explanation of the generated program

[1400] 1. User authentication:

[1401] Terminal: The user turns on the smart glasses and enters their ID and password. The entered information is sent to the server.

[1402] Server: Checks the received user information in the database and generates an authentication result. If authentication is successful, generates a token and sends it to the terminal.

[1403] Terminal: Displays the authentication result to the user.

[1404] 2. Information gathering and interviews:

[1405] User (field operation team): Collects on-site information using the smart glasses' camera and voice input (e.g., "The conveyor belt has stopped").

[1406] Terminal: Sends the entered information to the server.

[1407] Server: The AI ​​analysis engine processes the received information and classifies it in detail.

[1408] Server: Stores organized information in a database and displays it on a dashboard.

[1409] 3. Automated response to inquiries:

[1410] User (Field Team): Asks the AI ​​for specific information via smart glasses (e.g., "Where is the manual for this device?").

[1411] Terminal: Sends the query to the server.

[1412] Server: The AI ​​analysis engine analyzes the received inquiry, compares it with the database and the latest information, and generates an appropriate answer.

[1413] Server: Sends the generated answer to the user's terminal and displays it to the user.

[1414] 4. Emotion Recognition and Feedback:

[1415] User (field team or control team): Expresses emotions through tone of voice and text input.

[1416] Terminal: Sends emotion information to the server.

[1417] Server: The emotion recognition engine analyzes this and identifies the emotion.

[1418] Server: Generates appropriate feedback, sends it to the user's device, and displays it (e.g., "Please stay calm, our support team will assist you").

[1419] Specific examples

[1420] For example, if a problem occurs in a factory where the conveyor belt has stopped, a worker on the field team can use smart glasses to capture the situation with a camera and report the situation by voice, saying, "The conveyor belt has stopped." This information is sent to the server, where the AI ​​analysis engine analyzes it and classifies it as "Location: Conveyor belt, Status: Stopped," and displays it on a dashboard. The control team can then quickly issue instructions by looking at the dashboard. Furthermore, if the worker's trembling voice is identified as "tension" by the emotion recognition engine, feedback such as "Please stay calm and we will contact the technical support team" is automatically returned.

[1421] Example prompt sentence:

[1422] "A worker in a factory uses smart glasses to report a machine problem. Write the steps to resolve this problem. Also, include an action that recognizes the worker's emotions and provides appropriate feedback."

[1423] This allows the field team and the control team to share information quickly and accurately and take immediate action.It also responds to the emotions of workers, providing a safer and more efficient working environment.

[1424] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1425] Step 1: User authentication

[1426] Input: The user turns on the smart glasses and enters their ID and password.

[1427] Terminal operation: The entered ID and password are sent to the server.

[1428] Server operation: The received user information is checked against the database, and an authentication result is generated. If authentication is successful, an authentication token is generated and sent to the terminal.

[1429] Output: The terminal displays the authentication result to the user.

[1430] Step 2: Information gathering and interviews

[1431] Input: The user (field team) takes pictures of the scene with the camera on the smart glasses and inputs information by voice. For example, they report that the conveyor belt has stopped.

[1432] Device operation: Sends the user's voice input and camera images to the server.

[1433] Server operation: The AI ​​analysis engine processes the received information and classifies it in detail. For example, "Location: conveyor belt, status: stopped."

[1434] Output: The server stores the organized information in a database and displays it on a dashboard.

[1435] Step 3: Automate your inquiries

[1436] Input: A user (field team) queries the AI ​​for specific information via smart glasses, for example, "Where is the manual for this device?"

[1437] Terminal operation: Sends the user's inquiry to the server.

[1438] Server operation: The AI ​​analysis engine analyzes the received inquiry, compares it with the database and the latest information, and generates an appropriate answer.

[1439] Output: The server sends the generated answer to the field team's terminal and displays it to the user.

[1440] Step 4: Emotion recognition and feedback

[1441] Input: Users (field or control teams) express emotions through tone of voice or text input.

[1442] Device operation: Sends user's voice and text information to the server.

[1443] Server operation: The emotion recognition engine analyzes this and identifies the user's emotion. For example, it recognizes a trembling voice as "tension."

[1444] Output: The server generates appropriate feedback (e.g., "Please stay calm, we will contact our technical support team") and sends it to the user's device for display.

[1445] Step 5: Organize and share the overall information

[1446] Input: All information aggregated on the server.

[1447] Server operation: The server periodically aggregates all information, and the AI ​​analysis engine systematically organizes it.

[1448] Output: The organized information is visually displayed on a dashboard, and real-time notifications are sent to the user's device whenever the information is updated.

[1449] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1450] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1451] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1452] [Fourth embodiment]

[1453] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1454] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1455] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1456] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1457] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1458] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1459] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1460] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1461] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1462] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1463] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1464] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1465] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1466] The present invention is a system for facilitating information sharing between the control team and the field activity team. The program processing in this system will be explained below.

[1467] The system consists of devices used by the control team and field teams, and a central server that manages information. The server uses AI to receive, analyze, organize, and share appropriate information. The system also sends real-time notifications to user devices, ensuring that users always have access to the latest information.

[1468] Program processing flow (natural language explanation)

[1469] 1. User Authentication

[1470] Device: The user launches the app and enters their ID and password on the login screen.

[1471] Terminal: Sends the entered ID and password to the server.

[1472] Server: Checks the received user information in the database and returns the authentication result to the terminal.

[1473] Terminal: If authentication is successful, grant the user access to the dashboard.

[1474] 2. Information gathering and interviews

[1475] User (on-site action team): Reports the situation on-site in chat format (e.g., "The building has collapsed. 10 people injured.").

[1476] Terminal: Sends user input to the server.

[1477] Server: Receives information, and the AI ​​analyzes and organizes it into detailed items (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10.").

[1478] Server: Stores organized information in a database and displays it on a dashboard.

[1479] Terminal (control team): Organized information is displayed in real time on the dashboard.

[1480] 3. Automated response to inquiries

[1481] User (on-site response team): Asks the AI ​​for the necessary information (e.g., "Where is the nearest evacuation shelter?").

[1482] Terminal: Sends the query to the server.

[1483] Server: The AI ​​compares the database with the latest information and generates an appropriate answer (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away.").

[1484] Server: Sends the generated answers to the terminals of the field activity team.

[1485] Terminal (Field Team): The answer is displayed to the user.

[1486] 4. Organizing and sharing overall information

[1487] Server: All information is periodically aggregated and AI organizes it in a visually easy-to-understand manner.

[1488] Server: View overall information on the dashboard.

[1489] Server: Sends notifications to all user devices when information is updated.

[1490] Terminals (Control Team and Field Operations Team): View the dashboard and access real-time updated information.

[1491] Specific examples

[1492] For example, if building A collapses during a disaster, an operator from the local activity team can type "Building A collapsed, 10 injured" into chat on their smartphone. This information is sent to the server, where it is analyzed by AI and organized as "Location: Building A, Situation: Collapsed, Number of injured: 10." The organized information is displayed on the control team's dashboard, contributing to the rapid decision-making of restoration instructions.

[1493] Additionally, when a local activity team asks, "Where is the nearest evacuation shelter?" the server checks the database and replies, "It's at the community center, a 10-minute walk away," which is displayed on the local activity team's terminal.

[1494] This allows information to be shared smoothly between the field activity team and the control team, reducing misunderstandings and duplicate questions, and enabling efficient disaster response.

[1495] The processing flow will be explained below.

[1496] Specific processing steps of the program

[1497] User Authentication Process

[1498] Step 1:

[1499] Device: The user launches the app and enters their ID and password on the login screen.

[1500] Step 2:

[1501] Terminal: Sends the entered ID and password to the server.

[1502] Step 3:

[1503] Server: Compares the received ID and password with the database and generates an authentication result.

[1504] Step 4:

[1505] Server: If authentication is successful, generates an authentication token and sends it to the terminal. If authentication fails, sends an error message to the terminal.

[1506] Step 5:

[1507] Terminal: Displays the authentication result (success or failure) to the user.

[1508] Information gathering and interviews

[1509] Step 1:

[1510] User (on-site activity team): Enters the situation at the scene in chat format (e.g., "Building A has collapsed, 10 injured.").

[1511] Step 2:

[1512] Terminal: Sends user input to the server.

[1513] Step 3:

[1514] Server: The received information is analyzed and the AI ​​classifies the content in detail (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10.").

[1515] Step 4:

[1516] Server: Stores organized information in a database.

[1517] Step 5:

[1518] Server: Organized information is displayed in real time on a dashboard.

[1519] Step 6:

[1520] Terminal (control team): Organized information is displayed on a dashboard, allowing the control team to check the latest information.

[1521] Automated response to inquiries

[1522] Step 1:

[1523] User (Field Operations Team): Asks the AI ​​for specific information (e.g., "Where is the nearest evacuation shelter?").

[1524] Step 2:

[1525] Terminal: Sends the query to the server.

[1526] Step 3:

[1527] Server: The AI ​​analyzes the incoming inquiries and compares them with the latest information in the database.

[1528] Step 4:

[1529] Server: Generates an appropriate answer (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away.").

[1530] Step 5:

[1531] Server: Sends the generated answers to the terminals of the field activity team.

[1532] Step 6:

[1533] Terminal (Field Team): Displays the generated answers to the user.

[1534] Organizing and sharing overall information

[1535] Step 1:

[1536] Server: All collected information is periodically aggregated and systematically organized by AI.

[1537] Step 2:

[1538] Server: View organized overview information on the dashboard.

[1539] Step 3:

[1540] Server: Sends real-time notifications to user devices whenever information is updated.

[1541] Step 4:

[1542] Devices (Control and Field Operations Teams): Receive notifications and access the dashboard for the latest information.

[1543] Step 5:

[1544] Terminals (control team and field operation team): Detailed information can be retrieved from the dashboard as needed to help understand the situation on site and carry out rescue operations.

[1545] Example 1

[1546] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1547] There is a problem that information sharing between the field operation team and the control team is not smooth, and it takes a long time to share and analyze information. In addition, it is not possible to respond appropriately to questions and information gathering on the ground in real time, making it difficult to respond to disasters efficiently. Furthermore, there is a lack of visual organization of information and real-time notification of overall information, which makes it difficult for the field and command teams to act quickly.

[1548] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1549] In this invention, the server includes means for receiving information entered by a user and having AI analyze and organize the information, means for storing the organized information in a database and displaying it on a display means, means for the server to periodically aggregate all information and visually display the overall information, means for sending notifications to multiple user devices, means for AI to automatically respond to user inquiries, means for making the overall information accessible from user devices, and means for receiving user authentication information and returning the authentication result to the terminal. This allows information to be shared smoothly between the field activity team and the control team, enabling real-time information provision and response, and achieving efficient disaster response.

[1550] ---

[1551] "User" means a person who uses the system to enter or access information.

[1552] "Server" means a central management system for receiving, analyzing, organizing, storing, displaying, and sending notifications about information.

[1553] "Terminal" refers to a device used by a user, an electronic device that communicates with a server through an application.

[1554] "Information" refers to data such as on-site conditions and inquiries entered by the field activity team and control team.

[1555] "AI" stands for artificial intelligence, and refers to technology that analyzes incoming information, organizes it appropriately, and generates answers.

[1556] A "database" refers to a system for storing organized information and retrieving it as needed.

[1557] "Display means" refers to an interface for visually presenting organized information to a user.

[1558] "Aggregation" refers to the process of bringing together distributed information.

[1559] "Visually displaying" refers to presenting information in a format such as a graph or chart in an easy-to-read format to the user.

[1560] "Notification" refers to a message that informs a user device that new information exists.

[1561] An "inquiry" refers to an action in which a user requests information from a system.

[1562] "Authentication" refers to the process of verifying a user's ID and password and confirming that the user is a legitimate user.

[1563] ---

[1564] ---

[1565] The present invention is a system for facilitating information sharing between the control team and the field activity team. This system consists of terminals used by the control team and the field activity team, and a server that centrally manages information. Below, we will explain in detail how each component works.

[1566] First, a user accesses the system using a terminal. At this time, the user enters their ID and password on the login screen and sends the authentication information to the server. The server compares the received information with the database and returns the authentication result to the terminal. If authentication is successful, the user is allowed to access the dashboard.

[1567] Next, users from the field operation team use their devices to report on the situation at the scene. The users input information in chat format and send it to the server. The server uses AI to analyze the received information and organizes the results of the analysis. For example, if a report is made such as "Building collapsed. 10 injured," the AI ​​will organize this into detailed items such as "Location: Building A, Situation: Collapsed, Number of injured: 10." This organized information is stored in a database and displayed in real time on a dashboard.

[1568] Users can also inquire about the information they need on-site. For example, by asking the AI ​​a question such as "Where is the nearest evacuation shelter?", the AI ​​sends the question to the server. The server then uses the AI ​​to compare the latest information with the database and generate an appropriate answer. The generated answer is sent to the device of the on-site team and displayed to the user.

[1569] The server periodically aggregates all information, and AI organizes it visually for easy understanding. The organized overall information is displayed on a dashboard, and the server sends notifications to all user devices when the information is updated. This allows control teams and field teams to access the latest information in real time and respond quickly.

[1570] Specific examples

[1571] For example, if Building A collapses during a disaster, an operator from the field operation team can enter "Building A collapsed, 10 injured" into the chat app on their smartphone. This information is sent to the server, where the AI ​​analyzes and organizes it as "Location: Building A, Situation: Collapsed, Number of injured: 10," and stores it in a database. On the control team's device, this information is displayed in real time on a dashboard, helping them to quickly decide on recovery instructions.

[1572] In addition, when a field activity team inputs a question into their smartphone, such as "Where is the nearest evacuation shelter?", the server checks the database and generates the answer, "It's at the community center, a 10-minute walk away," which is displayed on the field activity team's device. This allows necessary information to be provided quickly, allowing field activities to proceed efficiently.

[1573] Example prompts for generative AI models

[1574] 1. "The field team reports that Building A has collapsed and 10 people are injured. I want to analyze and organize this information and display it on my dashboard."

[1575] 2. "In response to a question from the field team, 'Where is the nearest evacuation shelter?', please check the database and generate an appropriate answer."

[1576] These prompts facilitate collaboration between the server and the AI, enabling them to effectively utilize input information from the user.

[1577] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1578] Now, let's explain the processing flow of the system program by dividing it into specific processing steps, and add inputs, outputs, and specific actions to each step.

[1579] ---

[1580] Step 1:

[1581] Input: The user launches the app and enters their ID and password on the login screen.

[1582] Specific operation: The user starts the device, opens the app, and accesses the login screen. They enter their ID and password and click the "Login" button.

[1583] Output: The entered ID and password are sent from the terminal to the server.

[1584] Step 2:

[1585] Input: Login information (ID and password)

[1586] Specific operation: The terminal uses a secure protocol (e.g., TLS / SSL) to encrypt the entered ID and password and send them to the server.

[1587] Output: Login information received by the server

[1588] Step 3:

[1589] Input: Login information received by the server

[1590] Specific operation: The server connects to a database (e.g., MySQL, PostgreSQL), compares the received ID and password with the registered information in the database, generates an authentication result, and returns the result (success or failure) to the terminal.

[1591] Output: The authentication result is generated and returned to the terminal.

[1592] Step 4:

[1593] Input: Authentication result

[1594] Specific operation: The device receives the authentication result, and if successful, allows the user to access the dashboard. A message indicating authentication success is displayed, and the device transitions to the dashboard screen. If unsuccessful, an error message is displayed.

[1595] Output: User accesses dashboard or is returned to login screen again

[1596] Step 5:

[1597] Input: User (on-site activity team) inputs the situation on-site in chat format

[1598] Specific operations: A user of the field operation team enters on-site information using the chat interface on the device and clicks the "Send" button.

[1599] Output: The entered site conditions are sent from the terminal to the server.

[1600] Step 6:

[1601] Input: Site status information

[1602] Specific operation: The server uses AI (natural language processing technology) to analyze the received information and organize it into specific categories (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10."). The organized information is then stored in a database.

[1603] Output: The analyzed and organized information is saved in a database.

[1604] Step 7:

[1605] Input: Organized information

[1606] Specific operation: The server prepares the organized information for display on the dashboard and updates the dashboard.

[1607] Output: The dashboard is updated to show organized information

[1608] Step 8:

[1609] Input: User (on-site activity team) enters the inquiry details

[1610] Specific operation: A user from the field team uses the chat interface on the device to enter a question and press the "send" button.

[1611] Output: The query is sent to the server

[1612] Step 9:

[1613] Input: Inquiry details

[1614] How it works: The server uses AI to analyze the received inquiry and compares it with the database and the latest information. An appropriate answer is generated (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away."). The answer is then sent to the device.

[1615] Output: The generated answer is sent to the device.

[1616] Step 10:

[1617] Input: Answer

[1618] Specific behavior: The device displays the received response in the chat interface within the app.

[1619] Output: The answer is displayed to the user

[1620] Step 11:

[1621] Input: General information updated regularly

[1622] How it works: The server periodically aggregates all information, and AI organizes it visually for easy understanding. The organized information is displayed on a dashboard, and notifications are sent to all user devices when the information is updated.

[1623] Output: Overall information is displayed on the dashboard and notifications are sent when information is updated.

[1624] Step 12:

[1625] Input: Information update notification

[1626] What happens: All user devices receive notifications and access the dashboard.

[1627] Output: User accesses the latest global information

[1628] This will allow you to understand the processing flow of the entire system in concrete detail.

[1629] (Application example 1)

[1630] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1631] There is a problem with insufficient information sharing between the field activity team and the control team during emergencies, making it difficult to efficiently manage and coordinate information in real time. In particular, there is a need for a system that can respond quickly and accurately to reports and inquiries from the field and visually grasp the overall situation.

[1632] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1633] In this invention, the server includes means for receiving information entered by the field activity team and having an AI analyze and organize the information, means for storing the organized information in a database and displaying it on a dashboard, means for the server to periodically aggregate all information and visually display overall information, means for sending notifications to multiple user devices, means for the AI ​​to automatically respond to inquiries from the field activity team, means for making overall information accessible from user devices, means for user authentication, means for reporting emergencies in chat format, means for visually displaying analyzed information, means for using an AI model to compare data and generate responses in an emergency, means for transmitting the generated responses to user devices, and means for updating and displaying information on user devices in real time. This allows information to be shared quickly in real time between the field activity team and the control team, enabling efficient emergency response.

[1634] The "on-site activity team" is a team that actually collects information at the scene of an emergency or disaster and reports on it.

[1635] The "control team" is a central management team that issues instructions based on information provided by the field operation team.

[1636] "AI" is an abbreviation for artificial intelligence, a technology that analyzes data and automatically makes judgments and predictions based on the results.

[1637] A "server" is a computer system that centrally manages information and executes various processes.

[1638] "User devices" are terminals such as smartphones and tablets used by field teams and control teams.

[1639] A "dashboard" is a screen or interface that visually displays information and allows users to grasp the situation in real time.

[1640] A "database" is an information storage system that stores organized information and allows it to be searched and used as needed.

[1641] "Notifications" is a feature that sends information updates and important announcements to user devices.

[1642] "User authentication" is a procedure for verifying that a user is a legitimate user.

[1643] "Chat format" is a text-based communication method that allows for real-time conversation.

[1644] "Analysis" is the process of analyzing information and organizing it into a meaningful form.

[1645] "Visual display" means displaying information in an easy-to-understand manner using graphs, maps, etc.

[1646] "Data matching" is the process of comparing input information with existing data to see if there is a match.

[1647] "Answer generation" is a function that automatically creates an appropriate response based on the results of data matching.

[1648] "Real-time" means that information is processed and notified within a very short time after it is generated.

[1649] This invention is a system that allows field teams and control teams to share information quickly and accurately in an emergency. The system receives information entered by the field team and uses AI to analyze and organize it, allowing the control team to grasp the situation in real time.

[1650] The system mainly consists of a server and a user device.

[1651] Hardware and Software

[1652] Hardware:

[1653] Server (Central Information Management System)

[1654] Smartphones (devices for the field activity team and control team)

[1655] software:

[1656] Flask (Python framework)

[1657] AI module (for data analysis)

[1658] Server Processing

[1659] The server receives information reported by the field teams, which are then analyzed by AI. The analyzed information is organized into a visually easy-to-understand format and stored in a database. It is then displayed in real time on a dashboard. The AI ​​also automatically responds to inquiries from the field teams, generating appropriate answers and sending them to the field teams' user devices.

[1660] User Device Capabilities

[1661] The user devices are smartphones used by the field operation team and the control team. These devices perform user authentication and have the function of reporting emergencies in chat format. The reported information is sent to a server, and the analysis results are displayed on the user device in real time.

[1662] Specific examples

[1663] For example, if a field team reports using their smartphone that "Building B is on fire, 5 injured," the information is analyzed by the server and displayed in organized form as "Location: Building B, Situation: Fire, Number of injured: 5." The control team can view the dashboard and issue instructions quickly based on the latest information. Also, if the field team asks, "Where is the nearest evacuation shelter?" the AI ​​will check the database and answer, "The nearest evacuation shelter is the community center, a 10-minute walk away," and send this information to the field team's device.

[1664] Prompt Sentence Examples

[1665] "As an AI model for a safety management system, please analyze the following information: Building B is on fire, 5 injured."

[1666] In this way, information management and sharing during emergencies can be carried out quickly and accurately, enabling efficient responses.

[1667] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1668] Step 1:

[1669] The user starts up their smartphone device and enters their ID and password on the login screen. The device sends the entered ID and password to the server. The server checks the received user information in a database and returns the authentication result to the device. The input is the user ID and password, and the output is the authentication result. Data processing involves verifying the user information, thereby confirming that the user is legitimate.

[1670] Step 2:

[1671] The user (on-site action team) reports the status of the emergency in chat format. Specifically, they input text information such as "Building B is on fire, 5 injured." The device sends this report to the server. The server then gives the received information to the AI ​​for analysis. The input is the emergency report, and the output is the analyzed specific situation information. Data processing involves text analysis, which organizes the information.

[1672] Step 3:

[1673] The server stores the information analyzed by AI in a database. The organized information is then displayed in real time on a dashboard. The input is the information analyzed by AI, and the output is the information stored in the database and displayed on the dashboard. Data processing involves storing and visualizing the information, making it usable.

[1674] Step 4:

[1675] The local activity team queries the AI ​​for the necessary information. Specifically, they input the question, "Where is the nearest evacuation shelter?" The device sends this query to the server. The server's AI compares the latest information with the database and generates an appropriate answer. The input is the query, and the output is the generated answer. Data processing involves data comparison and answer generation, which provides appropriate information.

[1676] Step 5:

[1677] The server sends the generated response to the terminal of the field team, who then checks the response on their terminal. The input is the response information from the server, and the output is the response displayed on the user's terminal. Data processing involves sending and displaying the information, which allows the field team to obtain the necessary information.

[1678] Step 6:

[1679] The server periodically aggregates all information, and AI organizes it in a visually easy-to-understand format. Specifically, the overall information is displayed in the form of graphs and maps. The input is individual pieces of information data, and the output is overall information that has been visually organized. Data processing involves aggregating and visualizing the information, which allows the overall situation to be grasped.

[1680] Step 7:

[1681] Every time the information is updated, the server sends notifications to all user devices in real time. The input is the new updated information, and the output is the notification to the user device. The data processing is the notification generation and transmission, so that users receive the latest information in real time.

[1682] In this way, by specifically showing how input data is processed at each step and what output is obtained, it becomes easier to understand the flow of the entire system.

[1683] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1684] This invention is a system for smooth information sharing between the control team and the field activity team, and is combined with an emotion engine that recognizes the user's emotions. This system consists of devices used by the control team and the field activity team, and a central server that manages information. The server uses AI and the emotion engine to receive, analyze, organize, and share information as appropriate.

[1685] Program processing flow (natural language explanation)

[1686] 1. User Authentication

[1687] Device: The user launches the app and enters their ID and password on the login screen.

[1688] Terminal: Sends the entered ID and password to the server.

[1689] Server: Checks the received user information against the database and generates an authentication result.

[1690] Server: If authentication is successful, generates an authentication token and sends it to the terminal. If authentication fails, sends an error message to the terminal.

[1691] Terminal: Displays the authentication result (success or failure) to the user.

[1692] 2. Information gathering and interviews

[1693] User (on-site activity team): Enters the situation at the scene in chat format (e.g., "Building A has collapsed, 10 injured.").

[1694] Terminal: Sends user input to the server.

[1695] Server: The received information is analyzed and the AI ​​classifies the content in detail (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10.").

[1696] Server: Stores organized information in a database.

[1697] Server: Organized information is displayed in real time on a dashboard.

[1698] Terminal (control team): Organized information is displayed on a dashboard, allowing the control team to check the latest information.

[1699] 3. Automated response to inquiries

[1700] User (Field Operations Team): Asks the AI ​​for specific information (e.g., "Where is the nearest evacuation shelter?").

[1701] Terminal: Sends the query to the server.

[1702] Server: The AI ​​analyzes the incoming inquiries and compares them with the latest information in the database.

[1703] Server: Generates an appropriate answer (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away.").

[1704] Server: Sends the generated answers to the terminals of the field activity team.

[1705] Terminal (Field Team): Displays the generated answers to the user.

[1706] 4. Organizing and sharing overall information

[1707] Server: All collected information is periodically aggregated and systematically organized by AI.

[1708] Server: View organized overview information on the dashboard.

[1709] Server: Sends real-time notifications to user devices whenever information is updated.

[1710] Devices (Control and Field Operations Teams): Receive notifications and access the dashboard for the latest information.

[1711] Terminals (control team and field operation team): Detailed information can be retrieved from the dashboard as needed to help understand the situation on site and carry out rescue operations.

[1712] 5. Recognizing and responding to user emotions

[1713] User (field team or control team): Expresses emotions (e.g., tension, anger) when interacting with the system through tone of voice and input text.

[1714] Terminal: Sends the user's emotional information to the server.

[1715] Server: The emotion engine analyzes the user's tone of voice and text to identify emotions (e.g., if the voice is trembling, it is recognized as nervousness).

[1716] Server: Generates an appropriate response based on the identified emotion (e.g., a calming message to ease tension).

[1717] Server: Sends the generated response to the user device and displays it to the user.

[1718] Specific examples

[1719] For example, if building A collapses during a disaster, an operator from the local activity team can type "Building A collapsed, 10 injured" into the chat on their smartphone. This information is sent to the server, where it is analyzed by AI and organized as "Location: Building A, Situation: Collapsed, Number of injured: 10." The organized information is displayed on the control team's dashboard, contributing to the prompt issuing of recovery instructions.

[1720] Furthermore, when a field team asks, "Where is the nearest evacuation shelter?", the server checks the database and replies, "It's at the community center, a 10-minute walk away," which is displayed on the field team's terminal. If the operator's voice is trembling, the emotion engine recognizes this as "tension" and automatically generates a response such as, "Please stay calm, the evacuation shelter is nearby."

[1721] This allows for smooth sharing of information between the field activity team and the control team, reducing misunderstandings and duplicate questions, enabling efficient disaster response, and also responding to users' emotions.

[1722] The processing flow will be explained below.

[1723] Specific processing steps of the program

[1724] User Authentication Process

[1725] Step 1:

[1726] Device: The user launches the app and enters their ID and password on the login screen.

[1727] Step 2:

[1728] Terminal: Sends the entered ID and password to the server.

[1729] Step 3:

[1730] Server: Compares the received ID and password with the database and generates an authentication result.

[1731] Step 4:

[1732] Server: If authentication is successful, generates an authentication token and sends it to the terminal. If authentication fails, sends an error message to the terminal.

[1733] Step 5:

[1734] Terminal: Displays the authentication result (success or failure) to the user.

[1735] Information gathering and interviews

[1736] Step 1:

[1737] User (on-site activity team): Enters the situation at the scene in chat format (e.g., "Building A has collapsed, 10 injured.").

[1738] Step 2:

[1739] Terminal: Sends user input to the server.

[1740] Step 3:

[1741] Server: The received information is analyzed and the AI ​​classifies the content in detail (e.g., "Location: Building A, Situation: Collapsed, Number of injured: 10.").

[1742] Step 4:

[1743] Server: Stores organized information in a database.

[1744] Step 5:

[1745] Server: Organized information is displayed in real time on a dashboard.

[1746] Step 6:

[1747] Terminal (control team): Organized information is displayed on a dashboard, allowing the control team to check the latest information.

[1748] Automated response to inquiries

[1749] Step 1:

[1750] User (Field Operations Team): Asks the AI ​​for specific information (e.g., "Where is the nearest evacuation shelter?").

[1751] Step 2:

[1752] Terminal: Sends the query to the server.

[1753] Step 3:

[1754] Server: The AI ​​analyzes the incoming inquiries and compares them with the latest information in the database.

[1755] Step 4:

[1756] Server: Generates an appropriate answer (e.g., "The nearest evacuation shelter is the Civic Center, a 10-minute walk away.").

[1757] Step 5:

[1758] Server: Sends the generated answers to the terminals of the field activity team.

[1759] Step 6:

[1760] Terminal (Field Team): Displays the generated answers to the user.

[1761] Organizing and sharing overall information

[1762] Step 1:

[1763] Server: All collected information is periodically aggregated and systematically organized by AI.

[1764] Step 2:

[1765] Server: View organized overview information on the dashboard.

[1766] Step 3:

[1767] Server: Sends real-time notifications to user devices whenever information is updated.

[1768] Step 4:

[1769] Devices (Control and Field Operations Teams): Receive notifications and access the dashboard for the latest information.

[1770] Step 5:

[1771] Terminals (control team and field operation team): Detailed information can be retrieved from the dashboard as needed to help understand the situation on site and carry out rescue operations.

[1772] Recognizing and responding to user emotions

[1773] Step 1:

[1774] Users (field or control team): Express emotions (e.g., tension, anger) when interacting with the system through tone of voice and input text.

[1775] Step 2:

[1776] Terminal: Sends the user's emotional information to the server.

[1777] Step 3:

[1778] Server: The emotion engine analyzes the user's tone of voice and text to identify emotions (e.g., if the voice is trembling, it is recognized as nervousness).

[1779] Step 4:

[1780] Server: Generates an appropriate response based on the identified emotion (e.g., a calming message to ease tension).

[1781] Step 5:

[1782] Server: Sends the generated response to the user device and displays it to the user.

[1783] Example 2

[1784] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1785] When coordinating information between the field operation team and the control team, it is difficult to share accurate and appropriate information in real time. In addition, there is a need to recognize the emotions of users working in the field and respond appropriately, but current systems do not adequately fulfill this requirement.

[1786] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1787] In this invention, the server includes means for receiving information entered by the field activity team and analyzing and organizing the information using a machine learning model, means for storing the organized information in a database and displaying it on an information display device, means for a management device to periodically aggregate all information and visually display overall information, means for sending notifications to multiple user terminals, means for a generation engine to automatically respond to inquiries from the field activity team, means for analyzing tone of voice and input text using an emotion engine that recognizes user emotions and generating an appropriate response message, and means for making overall information accessible from user terminals. This enables accurate and appropriate information sharing in real time between the field activity team and the control team, and realizes flexible responses according to user emotions.

[1788] "Field activity teams" are personnel or teams that mainly operate on-site, reporting on the situation and gathering information.

[1789] "Machine learning models" refer to the algorithms and mathematical models used by the server to perform data analysis and pattern recognition.

[1790] "Means of organizing" refers to the function of receiving and analyzing information, classifying and arranging the content in a specific format, and storing and displaying it.

[1791] A "database" is an information system designed to efficiently store and manage organized information and enable quick retrieval when needed.

[1792] "Information display device" refers to equipment or software that provides an interface for visually checking and manipulating collected information.

[1793] "Management device" refers to a computer system that centrally manages the collection, organization, and display of all information.

[1794] "User terminal" refers to a device (e.g., smartphone, tablet, PC) used by the field activity team and control team that can input and output information.

[1795] "Means for sending notifications" refers to a function that allows the server to notify the user terminal of information updates and important events in real time.

[1796] "Generation engine" refers to AI technology that automatically generates appropriate answers to user inquiries.

[1797] An "emotion engine" refers to AI technology that analyzes emotions from the user's tone of voice and input text, and automatically generates a response appropriate to those emotions.

[1798] "Comprehensive information" refers to comprehensive information that integrates and organizes all data collected from field teams and other sources.

[1799] "Visual display means" refers to the function of displaying collected and organized information in the form of graphs, charts, text, etc. in a way that is easy to understand at a glance.

[1800] "Means of making it accessible" refers to the mechanism that provides the authority and functions for a user device to view and operate the entire information.

[1801] This invention is a system for facilitating information sharing between field teams and control teams, and is combined with an emotion engine that recognizes user emotions. This system consists of devices used by the field teams and control teams, and a central server that manages information. The server utilizes machine learning models and the emotion engine to receive, analyze, organize, share appropriate information, and respond to emotions.

[1802] The server is equipped with a database, machine learning model, and emotion engine. PostgreSQL is used as the database, and TensorFlow is used as the machine learning model. The emotion engine uses IBM Watson Tone Analyzer. Additionally, Grafana is used as the information display device, and Firebase Cloud Messaging (FCM) is used for the notification function.

[1803] Users of the field response team interactively input the situation at the scene. This information is sent to the server via their terminal. The server analyzes the received information using a machine learning model and categorizes and organizes it in detail. For example, an input such as "Building A collapsed, 10 injured" is organized as "Location: Building A, Situation: Collapsed, Number of injured: 10." The organized information is stored in a PostgreSQL database.

[1804] The server then displays the organized information in real time on the Grafana dashboard via the management device, allowing the control team to view the latest information on their terminals. At the same time, the server periodically aggregates all the information and provides a means for visually displaying it. Every time this information is updated, the server sends a real-time notification to the user's terminal.

[1805] Additionally, when the field team makes a direct inquiry to the server, such as "Where is the nearest evacuation shelter?", the server automatically generates an appropriate answer using a generative AI model (OpenAI GPT-4 model). The answer automatically generated is "It's at the community center, a 10-minute walk away," and is displayed on the field team's terminal.

[1806] Furthermore, an emotion engine (IBM Watson Tone Analyzer) analyzes emotions expressed by the user while interacting with the system, such as tension or anger, and automatically generates an appropriate response message. If the user is nervous, a message such as "Please stay calm, a shelter is nearby" is generated and sent to the user's device.

[1807] For example, when a disaster occurs, an operator from the field team can type "Building A has collapsed, 10 injured" into the chat on their smartphone. This information is analyzed and organized on the server and displayed on the control team's dashboard. If the field team asks, "Where is the nearest evacuation shelter?", the server will respond with "The community center, a 10-minute walk away." At the same time, if the operator is nervous, a message will be sent saying, "Please stay calm, the evacuation shelter is nearby."

[1808] Examples of prompts are:

[1809] "Building A has collapsed. 10 people are injured. Please tell me the nearest evacuation shelter."

[1810] "I want to see the current status of injured people in real time. How can I access the dashboard?"

[1811] "What kind of message would be effective for a user who is nervous?"

[1812] This enables accurate and appropriate information sharing in real time between the field operation team and the control team, and also enables flexible responses to user emotions.

[1813] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1814] Program processing flow and specific explanation of each processing step

[1815] Step 1: User authentication

[1816] 1. Input: The user enters the ID and password into the terminal.

[1817] 2. Operation: The device hashes the entered ID and password and sends the data to the server via secure communication (HTTPS).

[1818] 3. Server:

[1819] Data processing: The server receives the hashed user information and checks it against the MySQL database.

[1820] Data operation: Executes SQL queries to retrieve user information and generate authentication results.

[1821] 4. Output: If authentication is successful, a JWT (JSON Web Token) is generated and sent to the terminal. If authentication fails, an error message is generated and sent.

[1822] Step 2: Gathering information and conducting interviews

[1823] 1. Input: The user (on-site activity team) inputs the situation on the scene in chat format. Example: "Building A collapsed, 10 injured."

[1824] 2. Operation: The device sends the input information to the server via HTTPS.

[1825] 3. Server:

[1826] Data processing: The text information received by the server is analyzed using TensorFlow's machine learning model and classified in detail.

[1827] Data calculation: Classify the information as "Location: Building A, Situation: Collapsed, Number of injured: 10."

[1828] 4. Output: The classified information is stored in a PostgreSQL database and sent to an information display device (Grafana) in real time for display.

[1829] Step 3: Automate your inquiries

[1830] 1. Input: A user (field operations team) asks for specific information, e.g., "Where is the nearest evacuation shelter?"

[1831] 2. Operation: The device sends the query to the server via HTTPS.

[1832] 3. Server:

[1833] Data processing: The query received by the server is input into the OpenAI GPT-4 model for analysis.

[1834] Data calculations: AI compares databases with the latest information and generates appropriate answers.

[1835] 4. Output: The generated answer is sent to the device and displayed to the user. For example, "The nearest evacuation shelter is the Civic Center, a 10-minute walk away."

[1836] Step 4: Organize and share overall information

[1837] 1. Input: All collected information is collected on the server.

[1838] 2. Operation: The server periodically aggregates this information.

[1839] 3. Server:

[1840] Data processing: Information is systematically organized using deep learning models running on NVIDIA GPUs with CUDA acceleration.

[1841] Data calculation: Visually organize the overall information.

[1842] 4. Output: The organized overall information is displayed on a Grafana dashboard, and notifications are sent to the user's device using Firebase Cloud Messaging (FCM) every time the information is updated.

[1843] Step 5: Recognize and respond to user emotions

[1844] 1. Input: The user expresses their emotions through tone of voice and text while interacting with the system.

[1845] 2. Operation: The device sends voice and text information expressing emotions to the server via WebRTC.

[1846] 3. Server:

[1847] Data processing: The emotion engine (IBM Watson Tone Analyzer) analyzes the received voice and text to identify emotions.

[1848] Data Calculation: Based on the analysis results, an appropriate response message is generated.

[1849] 4. Output: Send the generated message to the terminal and display it to the user. Example: "Please stay calm, shelter is nearby."

[1850] This allows the system to process information quickly and accurately at each stage, enabling real-time information sharing between the field operation team and the control team, while also enabling it to flexibly respond to user emotions.

[1851] (Application example 2)

[1852] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1853] Conventional information sharing systems for on-site work have had issues such as time-consuming information organization and analysis, and difficulty in responding immediately in emergencies. Furthermore, the lack of feedback that takes into account the emotions and mental state of workers makes it difficult to create an efficient and safe work environment. This invention solves these issues by providing rapid and accurate information sharing and feedback that also takes into account the mental state of workers.

[1854] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving information entered by the field activity team and analyzing and organizing the information using AI, means for storing the organized information in a database and displaying it on a dashboard, and means for periodically aggregating information and visually displaying the overall information. This enables rapid organization and sharing of information. In addition, by using means for recognizing user emotions and providing appropriate feedback, a safe and efficient work environment for workers is ensured.

[1855] A "field unit" is a person or team whose role is to gather and report information at a specific site.

[1856] "AI" refers to the intelligent behavior and learning capabilities imitated by computer systems as artificial intelligence.

[1857] A "dashboard" is an interface for visually displaying collected information, allowing users to check the situation in real time.

[1858] "Server" means a central management system that receives, analyzes, stores, and notifies user devices of information.

[1859] "User device" refers to a terminal used by field teams and control teams, including smartphones, tablets, smart glasses, etc.

[1860] "Sending notifications" refers to the act of automatically sending updates or responses from the server to the user device.

[1861] An "emotion engine" is a technology that recognizes emotions by analyzing the user's tone of voice and input text.

[1862] "Feedback" is the act of providing appropriate responses or instructions according to the user's emotions and situation.

[1863] "Analyzing and organizing information" means that the AI ​​processes, classifies, and reconstructs the information entered by the field team into an easy-to-understand format.

[1864] This invention is a system that enables smooth information sharing between field teams and control teams, and combines an emotion engine that recognizes the user's emotions. The field teams are responsible for collecting and reporting information at specific sites, while the control teams are responsible for managing information centrally and issuing instructions.

[1865] Hardware and software used

[1866] Hardware:

[1867] Smart glasses (e.g. Microsoft HoloLens)

[1868] Server (e.g. AWS EC2)

[1869] software:

[1870] AI analysis engine (e.g. TensorFlow)

[1871] Emotion recognition engine (e.g. IBM Watson Tone Analyzer)

[1872] Database (e.g. MySQL)

[1873] Front-end frameworks (e.g., React.js)

[1874] Backend frameworks (e.g., Node.js)

[1875] Explanation of the generated program

[1876] 1. User authentication:

[1877] Terminal: The user turns on the smart glasses and enters their ID and password. The entered information is sent to the server.

[1878] Server: Checks the received user information in the database and generates an authentication result. If authentication is successful, generates a token and sends it to the terminal.

[1879] Terminal: Displays the authentication result to the user.

[1880] 2. Information gathering and interviews:

[1881] User (field operation team): Collects on-site information using the smart glasses' camera and voice input (e.g., "The conveyor belt has stopped").

[1882] Terminal: Sends the entered information to the server.

[1883] Server: The AI ​​analysis engine processes the received information and classifies it in detail.

[1884] Server: Stores organized information in a database and displays it on a dashboard.

[1885] 3. Automated response to inquiries:

[1886] User (Field Team): Asks the AI ​​for specific information via smart glasses (e.g., "Where is the manual for this device?").

[1887] Terminal: Sends the query to the server.

[1888] Server: The AI ​​analysis engine analyzes the received inquiry, compares it with the database and the latest information, and generates an appropriate answer.

[1889] Server: Sends the generated answer to the user's terminal and displays it to the user.

[1890] 4. Emotion Recognition and Feedback:

[1891] User (field team or control team): Expresses emotions through tone of voice and text input.

[1892] Terminal: Sends emotion information to the server.

[1893] Server: The emotion recognition engine analyzes this and identifies the emotion.

[1894] Server: Generates appropriate feedback, sends it to the user's device, and displays it (e.g., "Please stay calm, our support team will assist you").

[1895] Specific examples

[1896] For example, if a problem occurs in a factory where the conveyor belt has stopped, a worker on the field team can use smart glasses to capture the situation with a camera and report the situation by voice, saying, "The conveyor belt has stopped." This information is sent to the server, where the AI ​​analysis engine analyzes it and classifies it as "Location: Conveyor belt, Status: Stopped," and displays it on a dashboard. The control team can then quickly issue instructions by looking at the dashboard. Furthermore, if the worker's trembling voice is identified as "tension" by the emotion recognition engine, feedback such as "Please stay calm and we will contact the technical support team" is automatically returned.

[1897] Example prompt sentence:

[1898] "A worker in a factory uses smart glasses to report a machine problem. Write the steps to resolve this problem. Also, include an action that recognizes the worker's emotions and provides appropriate feedback."

[1899] This allows the field team and the control team to share information quickly and accurately and take immediate action.It also responds to the emotions of workers, providing a safer and more efficient working environment.

[1900] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1901] Step 1: User authentication

[1902] Input: The user turns on the smart glasses and enters their ID and password.

[1903] Terminal operation: The entered ID and password are sent to the server.

[1904] Server operation: The received user information is checked against the database, and an authentication result is generated. If authentication is successful, an authentication token is generated and sent to the terminal.

[1905] Output: The terminal displays the authentication result to the user.

[1906] Step 2: Information gathering and interviews

[1907] Input: The user (field team) takes pictures of the scene with the camera on the smart glasses and inputs information by voice. For example, they report that the conveyor belt has stopped.

[1908] Device operation: Sends the user's voice input and camera images to the server.

[1909] Server operation: The AI ​​analysis engine processes the received information and classifies it in detail. For example, "Location: conveyor belt, status: stopped."

[1910] Output: The server stores the organized information in a database and displays it on a dashboard.

[1911] Step 3: Automate your inquiries

[1912] Input: A user (field team) queries the AI ​​for specific information via smart glasses, for example, "Where is the manual for this device?"

[1913] Terminal operation: Sends the user's inquiry to the server.

[1914] Server operation: The AI ​​analysis engine analyzes the received inquiry, compares it with the database and the latest information, and generates an appropriate answer.

[1915] Output: The server sends the generated answer to the field team's terminal and displays it to the user.

[1916] Step 4: Emotion recognition and feedback

[1917] Input: Users (field or control teams) express emotions through tone of voice or text input.

[1918] Device operation: Sends user's voice and text information to the server.

[1919] Server operation: The emotion recognition engine analyzes this and identifies the user's emotion. For example, it recognizes a trembling voice as "tension."

[1920] Output: The server generates appropriate feedback (e.g., "Please stay calm, we will contact our technical support team") and sends it to the user's device for display.

[1921] Step 5: Organize and share the overall information

[1922] Input: All information aggregated on the server.

[1923] Server operation: The server periodically aggregates all information, and the AI ​​analysis engine systematically organizes it.

[1924] Output: The organized information is visually displayed on a dashboard, and real-time notifications are sent to the user's device whenever the information is updated.

[1925] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1926] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1927] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1928] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1929] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1930] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1931] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1932] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1933] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1934] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1935] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1936] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1937] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1938] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1939] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1940] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1941] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1942] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1943] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1944] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1945] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1946] The following is further disclosed regarding the above embodiment.

[1947] (Claim 1)

[1948] A means for AI to receive information entered by the field team and analyze and organize that information,

[1949] Next, we need a way to store the organized information in a database and display it on a dashboard.

[1950] A means for the server to periodically aggregate all information and visually display the overall information;

[1951] means for sending notifications to a plurality of user devices;

[1952] A means for AI to automatically respond to inquiries from field teams,

[1953] A system including means for making the global information accessible to a user device.

[1954] (Claim 2)

[1955] 2. The system according to claim 1, wherein a situation report from the field activity team is input in chat format and transmitted to the server.

[1956] (Claim 3)

[1957] 10. The system of claim 1, wherein the system sends a real-time notification to the user device whenever the global information is updated.

[1958] "Example 1"

[1959] ---

[1960] (Claim 1)

[1961] A means for AI to receive information entered by users and analyze and organize that information,

[1962] Next, a means for storing the organized information in a database and displaying it on a display means;

[1963] A means for the server to periodically aggregate all information and visually display the overall information;

[1964] means for sending notifications to a plurality of user devices;

[1965] A means for AI to automatically respond to user inquiries,

[1966] means for making the collective information accessible from a user device;

[1967] A system including means for receiving user authentication information and returning authentication results to a terminal.

[1968] (Claim 2)

[1969] 2. The system according to claim 1, wherein a user's status report is input in a chat format and transmitted to the server.

[1970] (Claim 3)

[1971] 10. The system of claim 1, wherein the system sends a real-time notification to the user device whenever the global information is updated.

[1972] ---

[1973] "Application Example 1"

[1974] (Claim 1)

[1975] A means for AI to receive information entered by the field team and analyze and organize that information,

[1976] Next, we need a way to store the organized information in a database and display it on a dashboard.

[1977] A means for the server to periodically aggregate all information and visually display the overall information;

[1978] means for sending notifications to a plurality of user devices;

[1979] A means for AI to automatically respond to inquiries from field teams,

[1980] means for making the collective information accessible from a user device;

[1981] means for performing user authentication;

[1982] A means to report emergencies via chat,

[1983] a means for visually displaying the analyzed information;

[1984] A means of using AI models to collate data and generate answers in emergencies; and

[1985] means for transmitting the generated answer to the user device;

[1986] A system including a means for updating and displaying information in real time on a user device.

[1987] (Claim 2)

[1988] 2. The system according to claim 1, wherein a situation report from the field activity team is input in chat format and transmitted to the server.

[1989] (Claim 3)

[1990] 10. The system of claim 1, wherein the system sends a real-time notification to the user device whenever the global information is updated.

[1991] "Example 2: Combining Emotion Engines"

[1992] (Claim 1)

[1993] A means to receive information entered by the field team and analyze and organize that information using a machine learning model;

[1994] Next, a means for storing the organized information in a database and displaying it on an information display device;

[1995] a means for the management device to periodically aggregate all information and visually display the overall information;

[1996] means for sending notifications to a plurality of user terminals;

[1997] A means for the generation engine to automatically respond to inquiries from field teams,

[1998] A means for analyzing the tone of voice and input text using an emotion engine that recognizes the user's emotions and generating an appropriate response message;

[1999] A system including means for making the global information accessible from a user terminal.

[2000] (Claim 2)

[2001] 2. The system according to claim 1, wherein a situation report from the field activity team is inputted in an interactive format and transmitted to the management device.

[2002] (Claim 3)

[2003] 10. The system of claim 1, wherein a notification is sent to the user terminal in real time whenever the overall information is updated.

[2004] "Application example 2 when combining emotion engines"

[2005] (Claim 1)

[2006] A means for AI to receive information entered by the field team and analyze and organize that information,

[2007] Next, we need a way to store the organized information in a database and display it on a dashboard.

[2008] A means for the server to periodically aggregate all information and visually display the overall information;

[2009] means for sending notifications to a plurality of user devices;

[2010] A means for AI to automatically respond to inquiries from field teams,

[2011] means for making the collective information accessible from a user device;

[2012] A system that includes a means for recognizing a user's emotions and providing appropriate feedback.

[2013] (Claim 2)

[2014] 2. The system according to claim 1, wherein a situation report from the field activity team is input in chat format and transmitted to the server.

[2015] (Claim 3)

[2016] 10. The system of claim 1, wherein the system sends a real-time notification to the user device whenever the global information is updated. [Explanation of symbols]

[2017] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for AI to receive information entered by the field team and analyze and organize that information, Next, we need a way to store the organized information in a database and display it on a dashboard. A means for the server to periodically aggregate all information and visually display the overall information; means for sending notifications to a plurality of user devices; A means for AI to automatically respond to inquiries from field teams, A system including means for making the global information accessible to a user device.

2. 2. The system according to claim 1, wherein a situation report from the field activity team is input in chat format and transmitted to the server.

3. The system of claim 1 , wherein the system sends a real-time notification to the user device whenever the global information is updated.

Images