System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving information transmitted from a user terminal; means for filtering the received information to exclude unnecessary information; means for analyzing the filtered information and clustering related information; means for extracting keywords from the clustered information and generating a brief description; means for identifying a missing portion in the generated description and transmitting additional information to the user; means for receiving the additional information and regenerating a final description; and means for distributing the final description to other users.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Current social networking sites are often filled with a lot of unnecessary and false information, making it extremely time-consuming for users to search for the information they need. Particularly during events or disasters, it is difficult to quickly and concisely obtain reliable information in real time. This can leave many users confused and anxious. Furthermore, there is a lack of means for efficiently gathering necessary additional information while ensuring the reliability of the information. The purpose of the present invention is to solve these problems. [Means for solving the problem]

[0005] The present invention provides a means for receiving information sent from a user terminal, a means for filtering the received information to eliminate unnecessary information, and a means for analyzing the filtered information and clustering related information. It also provides a means for extracting keywords from the clustered information and generating a concise description, and a means for identifying deficiencies in the generated description and sending additional information to the user. It also includes a means for receiving the additional information, regenerating a final description, and distributing the final description to other users. Furthermore, since the posted information from the user terminal includes location information and a timestamp, the generation AI can extract keywords based on the clustered information and generate a description, allowing users to quickly and concisely obtain reliable information in real time.

[0006] "User terminal" refers to an electronic device that allows a user to input and transmit information, including a smartphone, tablet, or computer.

[0007] "Means for receiving information" refers to functions and protocols for receiving information sent from a user terminal.

[0008] "Means for filtering out unnecessary information" refers to the ability to analyze received information and remove meaningless data or irrelevant information based on specific criteria.

[0009] "Means for analyzing and clustering related information" refers to a function for further analyzing the filtered information and grouping together highly related information.

[0010] "Means for extracting keywords and generating concise descriptions" refers to the function of selecting important words from the clustered information and creating short, summarized descriptions based on those words.

[0011] "Means for identifying missing parts and sending additional information to the user" refers to the function of finding missing information in the generated explanation and sending questions to the user to complete it.

[0012] "Means for receiving additional information and regenerating the final description" refers to a function for receiving additional information from the user and then regenerating a more complete description by reflecting that information.

[0013] The "means for distributing the final description to other users" refers to a function for widely disseminating the final description to other users.

[0014] "Location Information" means data that indicates a user's current location, including GPS information and information obtained through other location-finding technologies.

[0015] A "timestamp" refers to data that indicates the date and time that particular information was created or transmitted.

[0016] "Generative AI" refers to artificial intelligence technology that extracts keywords and generates descriptions based on clustered information. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The system of the present invention is composed of a user terminal, a server, and a generation AI. Below, we will provide a detailed explanation of how each element operates and how the entire system functions.

[0039] Operation on the user device

[0040] Enter and submit information

[0041] When a user is at an event or disaster site, they open a dedicated application. They write and attach information about the situation at the site in the form of text, images, or videos, and then press the send button. For example, they could post something like, "Band X is currently performing on Stage B, and a large audience is gathering." At this time, location information and a timestamp are automatically added from the user's device.

[0042] Processing on the server

[0043] Receiving and filtering information

[0044] The server receives information sent from the user's device. The received information includes the content of the post, location information, timestamp, etc. After receiving the information, the server filters it using a specific algorithm to eliminate unnecessary or irrelevant information. The relevant information is then passed on to the next processing step.

[0045] Information clustering and keyword extraction

[0046] The filtered information is analyzed by a generation AI in the server, which clusters the filtered information into related groups (e.g., all information related to Stage B into one cluster).The generation AI then extracts important keywords from each cluster and generates a concise description.

[0047] Examining the generated explanation and generating follow-up questions

[0048] The server then carefully examines the generated description to identify any missing information or ambiguities. For example, if the generated description is "Stage B is currently crowded because band X is playing," the server will automatically generate additional questions about the situation on other stages and the number of audience members.

[0049] Asking the user additional questions and completing information

[0050] Submit and answer additional questions

[0051] The server generates a follow-up question and sends it to the user. For example, a question like "What is the status of the other stages?" is sent to the user. The user responds to the question by providing additional information in the form of text or images. This answer is then sent back to the server.

[0052] Final information generation and distribution

[0053] Re-aggregating information and regenerating descriptions

[0054] The server receives additional information from the user and stores it in a database. The generation AI then regenerates the final description based on the newly added information. For example, a specific description such as "Currently, band X is performing on stage B, causing congestion. Band Y is preparing on stage C." is generated.

[0055] Final information distribution

[0056] The server then distributes the resulting descriptions to other users in real time, who then display the information as notifications or feeds.

[0057] Specific examples

[0058] Music festival scenario

[0059] 1. User A posts at a music festival, "Band X is performing on Stage B."

[0060] 2. The server receives this post and clusters it with other related posts.

[0061] 3. The AI ​​generates a description like "Stage B is currently crowded because band X is performing," but it lacks information about the other stages.

[0062] 4. The server sends a follow-up question to User A: "What's the status of the other stages?"

[0063] 5. User A answers, "Band △△ is preparing for stage C."

[0064] 6. The final description will be "Currently, stage B is crowded as band X is performing. band YY is preparing on stage C."

[0065] Disaster scenarios

[0066] 1. User B posts, "The roads are cracked due to the earthquake."

[0067] 2. The server receives this post and parses it along with other related posts.

[0068] 3. The AI ​​generates the description "Roads are cracked due to the earthquake. Aftershocks are continuing," but it lacks information about evacuation sites.

[0069] 4. The server asks User B an additional question: "Where is the evacuation site?"

[0070] 5. User B answers, "The evacuation site is a nearby park."

[0071] 6. Finally, the following message is displayed: "Roads are cracked due to the earthquake. Aftershocks are continuing. Evacuation shelter is in a nearby park."

[0072] The system of the present invention allows users to obtain fast and reliable information in real time, enabling accurate situational understanding even during events or disasters.

[0073] The processing flow will be explained below.

[0074] Step 1:

[0075] The user opens the dedicated application, writes and attaches the situation at the site in the form of text, images, videos, etc., and presses the send button. Location information and a timestamp are automatically added from the user's device.

[0076] Step 2:

[0077] The server receives the information sent from the user terminal, including the posted text, images, videos, location information, and timestamps.

[0078] Step 3:

[0079] The server filters the received information using specific algorithms to remove unnecessary or irrelevant information, and stores the filtered information for analysis.

[0080] Step 4:

[0081] Based on the filtered information, the generative AI clusters this information, specifically grouping highly related information together.

[0082] Step 5:

[0083] From the clustered information, the generative AI extracts important keywords, and generates an initial, concise description based on the extracted keywords.

[0084] Step 6:

[0085] The server reviews the generated description to identify any missing or ambiguous information, confirms the additional information needed, and proceeds to the next step.

[0086] Step 7:

[0087] The server automatically generates additional questions based on the missing information and sends the generated questions to the corresponding user terminal.

[0088] Step 8:

[0089] The user terminal receives the additional question, and the user provides an answer (text, image, etc.) to the question. The user presses the send button to send the answer to the server.

[0090] Step 9:

[0091] The server receives additional information from the user and passes it back to the AI ​​for re-analysis, which then generates the final description.

[0092] Step 10:

[0093] The server then distributes the final description to other users, who then display it in real time as notifications or feeds on their devices.

[0094] This allows users to obtain fast and accurate information in real time.

[0095] Example 1

[0096] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0097] During modern events and disasters, it is important to quickly and accurately grasp the situation at the scene, but manually collecting and organizing information takes time and effort. It is also difficult to ensure the reliability of the information and real-time availability of the information. Furthermore, when efficiently processing multi-format data (text, images, and video) and converting it into easy-to-understand descriptions, additional information from users is often not reflected in a timely manner. To address these issues, effective information reception, filtering, clustering, keyword extraction, and improved accuracy of automatically generated descriptions are required.

[0098] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0099] In this invention, the server includes means for receiving multi-format data transmitted from a user terminal, means for initially processing the received data and eliminating irrelevant data, means for clustering the filtered data and organizing it into related information groups, means for extracting keywords from the clustered information groups and generating an explanatory text, means for identifying missing information in the generated explanatory text and requesting additional information from the user, means for receiving the additional information from the user and regenerating a final explanatory text, and means for distributing the final explanatory text to other users in real time, thereby enabling efficient processing of multi-format data and providing reliable information in real time.

[0100] A "user terminal" is a computing device that a user uses to input and transmit information, and includes smartphones, tablets, laptops, and the like.

[0101] "Multi-format data" refers to digital data in different formats, such as text, images, and video.

[0102] A "server" is a computer system that receives, processes, stores, and distributes data sent from user terminals over a network.

[0103] "Initial processing" refers to processing to analyze received data and eliminate signal noise and unnecessary information.

[0104] "Filtering" refers to the process of removing unnecessary information from received data and extracting highly relevant information.

[0105] "Clustering" is a technical method for grouping data that have similar characteristics.

[0106] An "information group" is a collection of a series of clustered related information.

[0107] "Keyword extraction" is the process of identifying and extracting important words and phrases from text data.

[0108] "Explanation" refers to text information generated in a format that is easy for users to understand.

[0109] A "generative AI model" is an algorithm or model that uses artificial intelligence to analyze data, extract keywords, and generate descriptions.

[0110] "Additional information" refers to information requested from the user to reacquire data that was missing at the time of initial information collection.

[0111] "Real-time delivery" is the process of providing the results to other users immediately at the moment the data is collected and analyzed.

[0112] The system of the present invention is composed of a user terminal, a server, and a generative AI model. Below, we will explain in detail how each element operates and how the entire system functions.

[0113] Hardware and Software Configuration

[0114] User device:

[0115] A user device is a computing device used by a user, and includes smartphones, tablets, laptops, etc. Using these devices, users open a dedicated application and enter information about an event or disaster site in the form of text, images, and videos. For example, a user might enter text such as "Band X is currently performing on Stage B," attach a photo, and press the send button. The user device automatically adds location information and a timestamp before sending the information.

[0116] server:

[0117] The server is a computer system that receives and processes multi-format data sent from user devices via a network. The received data is stored in a real-time database, and irrelevant and inconsistent data is eliminated as an initial process. The server then filters the data using a specific algorithm to extract only highly relevant data. The filtered data is then clustered using a generative AI model and organized into related information groups. Machine learning techniques are used for this clustering process. The generative AI model then extracts keywords from each cluster and generates a concise explanation to be communicated to the user.

[0118] Generative AI models:

[0119] The generative AI model is an algorithm that extracts keywords from the clustered information and generates explanatory text. If the generated explanation is incomplete, the server identifies this and generates and sends a prompt to the user requesting additional information.

[0120] Specific examples

[0121] Music festival scenario

[0122] 1. User A is at a music festival and types in the text "XX band is performing on Stage B," attaches a photo of the stage, and presses the send button.

[0123] 2. The server receives this information and stores the data, including location and timestamp, in a real-time database.

[0124] 3. A filtering algorithm is applied to remove unnecessary data.

[0125] 4. The generative AI model clusters related information and combines information related to "Stage B" into a single cluster.

[0126] 5. The generative AI model extracts keywords from the cluster and generates a description such as, "Stage B is crowded because band X is performing."

[0127] 6. The server generates a follow-up question: "What is the status of the other stages?" and notifies User A.

[0128] 7. User A replies, "Band △△ is being prepared on Stage C," and the information is sent again.

[0129] 8. The server receives the additional information, and the generative AI model regenerates the final description: "Currently, stage B is crowded because band X is performing. On stage C, band YY is preparing."

[0130] 9. The server distributes the final information to other users in real time.

[0131] Disaster scenarios

[0132] 1. User B posts, "The roads are cracked due to the earthquake," attaches a photo of the scene, and presses the send button.

[0133] 2. The server receives this information and stores it in a real-time database.

[0134] 3. A filtering algorithm is applied to extract only the required data.

[0135] 4. The generative AI model clusters related information and combines information about "earthquakes" into one cluster.

[0136] 5. The generative AI model extracts keywords from the cluster and generates a description such as, "The roads are cracked due to the earthquake. Aftershocks are continuing."

[0137] 6. The server generates an additional question, "Where is the evacuation site?" and notifies User B.

[0138] 7. User B replies, "The evacuation site is a nearby park," and the information is sent again.

[0139] 8. The server receives the additional information, and the generative AI model regenerates the final description: "Roads are cracked due to the earthquake. Aftershocks are continuing. Evacuation shelters are in nearby parks."

[0140] 9. The server distributes the final information to other users in real time.

[0141] This allows users to obtain fast and reliable information in real time, enabling accurate situational understanding even during events or disasters.

[0142] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0143] Step 1:

[0144] The user opens a dedicated application. The user enters information about the event or disaster site in text, image, and video format, and presses the "Send" button. At this time, the user's device automatically adds location information and a timestamp. A specific example of how this works is that the user enters "Band X is currently performing on Stage B," attaches a photo of the scene, and presses the "Send" button. The input data consists of text, images, location information, and a timestamp, and is then sent to the server.

[0145] Step 2:

[0146] The server receives multi-format data sent from user devices. The server performs initial processing of the received data and applies filtering algorithms to remove irrelevant or inconsistent data. The input data is all the information sent by the user, and the output data is clean data with unnecessary data removed. Specific operations include removing spam messages and checking format consistency.

[0147] Step 3:

[0148] The server clusters the filtered data. Using a generative AI model, related information is grouped and organized into related information clusters. The input data is the filtered data, and the output data is the clustered information clusters. Specifically, multiple posts about the same event are grouped into one cluster.

[0149] Step 4:

[0150] The server uses a generative AI model to extract keywords from the clustered information and generate a description. The input data is the clustered information, and the output data is the generated description. For example, a description such as "Stage B is crowded because band X is performing" is automatically generated.

[0151] Step 5:

[0152] The server examines the generated description and identifies any missing or ambiguous information. Based on this, the server generates a prompt to request additional information from the user. The input data is the generated description, and the output data is a prompt for additional information. Specifically, a question such as "What is the status of the other stages?" is generated.

[0153] Step 6:

[0154] The server notifies the user terminal of the generated prompt text. The user receives the notification and inputs additional information. This additional information is also sent in the form of text, images, or video. The input data is the additional information sent by the user, and the output data is additional data including the additional information. A specific operation is for the user to reply, "△△ band is being prepared on stage C," and send it.

[0155] Step 7:

[0156] The server receives additional information from the user and regenerates the final description using the generative AI model. The input data is the additional information from the user, and the output data is the regenerated final description. A specific example of how this works is that the final description might read, "Currently, band X is performing on stage B, causing congestion. Band Y is preparing on stage C."

[0157] Step 8:

[0158] The server distributes the generated final description to other users in real time. The user's device displays the final information in the form of a notification or feed. The input data is the final description, and the output data is the distributed notification. Specifically, the smartphone notification screen will display "Currently, band X is performing on stage B, causing congestion. Band Y is preparing on stage C."

[0159] This allows the system to efficiently process information from users and provide other users with reliable information in real time.

[0160] (Application example 1)

[0161] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0162] In modern society, it is extremely important to quickly and accurately collect information about abnormal situations and emergencies and notify relevant parties. However, current information collection systems often contain a lot of unnecessary information and lack real-time capabilities, making it difficult to respond quickly. Information is also often insufficient or ambiguous, making it difficult to collect additional information. This can hinder appropriate responses in emergencies. It is necessary to solve these problems and achieve efficient and rapid reporting and response to abnormal situations.

[0163] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0164] In this invention, the server includes: means for receiving information transmitted from a user terminal; means for filtering the received information to eliminate unnecessary information; means for analyzing the filtered information and clustering relevant information; means for extracting keywords from the clustered information and generating a concise explanation; means for identifying deficiencies in the generated explanation and transmitting additional information to the user; means for receiving the additional information and regenerating a final explanation; means for distributing the final explanation to other users; means for comprehensively analyzing the anomaly report information transmitted from the user and detecting special circumstances; and means for automatically generating follow-up questions based on the special circumstances and transmitting them to the user in real time. This enables the server to quickly and accurately collect and analyze anomaly information reported by users and efficiently supplement missing information. The generated information can then be instantly distributed to relevant parties, enabling a rapid response.

[0165] A "user terminal" is an electronic device operated by a user, and is a terminal device for inputting and transmitting information.

[0166] "Information" refers to all data sent from a user terminal, such as text, images, videos, location information, and timestamps.

[0167] "Means for receiving" refers to a mechanism for receiving and storing information sent from a user terminal.

[0168] A "filtering means" is a mechanism for filtering out unnecessary or irrelevant information from received information.

[0169] The "means for analyzing and clustering related information" is a method for analyzing the filtered information and grouping related information into one group.

[0170] "Means for extracting keywords and generating concise descriptions" is a technology for extracting important words and phrases from clustered information and creating concise sentences based on them.

[0171] The "means for identifying missing parts of the generated description and sending additional information to the user" is a mechanism for verifying the generated description, identifying missing information, and inquiring about that information from the user.

[0172] The "means for receiving additional information and regenerating the final description" is a method for creating a new, compiled final description based on the additional information provided by the user.

[0173] The "means for distributing the final description to other users" is a mechanism for transmitting the generated final description to other users in real time.

[0174] "Anomaly report information" refers to information reported by a user regarding an abnormality or emergency.

[0175] The "means for detecting special situations" is a method for analyzing abnormality report information and identifying abnormal situations based on specific conditions or patterns.

[0176] The "means for automatically generating follow-up questions and sending them to the user in real time" is a mechanism for generating necessary follow-up questions based on the detected abnormal situation and sending them to the user immediately.

[0177] The system of the present invention is composed of a user terminal, a server, and a generation AI. Below, we will provide a detailed explanation of how each element operates and how the entire system functions.

[0178] Operation on the user device

[0179] Enter and submit information:

[0180] Users use a dedicated application installed on their smartphones or smart glasses. When they witness an abnormal or emergency situation, they record the situation on the scene in the form of text, images, or videos, and press the send button. At this time, location information and a timestamp are automatically added. For example, a user might post, "I spotted a suspicious person. I'm attaching a photo."

[0181] Processing on the server

[0182] Receiving and filtering information:

[0183] The server receives the information sent from the user's device and uses the server's computing resources (e.g., AWS, Google Cloud Platform) and a filtering algorithm to remove unnecessary data from the received information. For example, it performs processing such as "excluding information about suspicious individuals."

[0184] Information clustering and keyword extraction:

[0185] The filtered information is analyzed by a generative AI (e.g., a GPT model) on the server. The generative AI uses a clustering algorithm to group related information together. It then extracts important keywords and generates a concise description. For example, it creates a description such as, "A suspicious person has been spotted in a specific location. We would like to report the current situation."

[0186] Examining the generated explanation and generating follow-up questions:

[0187] The server then examines the generated description to identify any missing information or ambiguity. For example, if the generated description is "A suspicious person was spotted in a specific location," the server automatically generates follow-up questions such as "What is the situation in other areas?". The server uses a generative AI model and prompts to generate follow-up questions.

[0188] Asking the user additional questions and completing information

[0189] Submit and answer additional questions:

[0190] The server generates a follow-up question and sends it to the user. For example, the user might ask, "What is the situation in other areas?" The user then provides additional information in the form of text or images. This answer is then sent back to the server.

[0191] Final information generation and distribution

[0192] Re-aggregate information and regenerate descriptions:

[0193] The server receives additional information from the user and stores it in a database. The AI ​​then regenerates the final description based on the newly added information. For example, a specific description such as "A suspicious person was spotted in a specific location. Other areas are being investigated."

[0194] Final information distribution:

[0195] The server then distributes the resulting description to other users in real time, and the user's device displays this information as a notification or feed, enabling fast and accurate information sharing.

[0196] Examples of concrete examples and prompts

[0197] Examples:

[0198] 1. A user reports that they have spotted a suspicious person in a specific location.

[0199] 2. The server receives the reports and filters and clusters the relevant information.

[0200] 3. The AI ​​generates a description that says "A suspicious person was spotted in a specific location," but lacks information about other areas.

[0201] 4. The server sends a follow-up question to the user: "What's the situation in other areas?"

[0202] 5. The user replies, "There are no abnormalities in other areas."

[0203] 6. The final description will be "Suspicious person spotted in specific location. No abnormalities in other areas."

[0204] Example prompt sentence:

[0205] A user reports a suspicious person in a specific location. Please suggest the next course of action based on the current situation on the scene.

[0206] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0207] Step 1:

[0208] Entering and sending information on the user's device

[0209] When a user witnesses an abnormal or emergency situation, they input information using a dedicated application installed on their smartphone or smart glasses. The input information is recorded in text, image, or video format and sent to a server by pressing the send button. Location information and a timestamp are automatically added.

[0210] Input: Text, images, videos, with automatically added location and timestamps

[0211] Output: Anomaly information sent to the server (text, images, videos, location information, timestamp)

[0212] Step 2:

[0213] Receiving and filtering information on the server

[0214] The server receives the anomaly information sent from the user terminal, and then uses a filtering algorithm to filter out unnecessary or irrelevant information.

[0215] Input: Anomaly information submitted by the user (text, image, video, location, timestamp)

[0216] Output: Filtered and relevant anomaly information

[0217] Step 3:

[0218] Server-based information clustering and keyword extraction

[0219] The filtered information is analyzed using generative AI (e.g., GPT model) to cluster relevant information, and then important keywords are extracted from the clustered information to generate a concise description.

[0220] Input: Filtered anomaly information

[0221] Output: Generated description (e.g. "A suspicious person was spotted at a specific location")

[0222] Step 4:

[0223] The server examines the generated explanation and generates follow-up questions

[0224] The server then examines the generated explanation to identify any missing information or ambiguity. It then automatically generates follow-up questions using a generative AI model and creates prompts to notify the user.

[0225] Input: Generated description

[0226] Output: Prompt for follow-up questions (e.g., "What about other areas?")

[0227] Step 5:

[0228] Sending and answering follow-up questions to users

[0229] The server generates a follow-up question and sends it to the user, who then provides additional information in the form of text, images, etc. The answer is then sent back to the server.

[0230] Input: Prompt for follow-up questions, additional information from the user (text, images, etc.)

[0231] Output: Additional information sent to the server

[0232] Step 6:

[0233] Server re-aggregates information and regenerates descriptions

[0234] The server receives additional information from the user and stores it in a database. The AI ​​then regenerates the final description based on the newly added information.

[0235] Input: Additional information from the user

[0236] Output: Final description (e.g. "Suspicious individuals were spotted in specific locations. No anomalies found in other areas.")

[0237] Step 7:

[0238] Final information generation and distribution

[0239] The server then distributes the resulting description to other users in real time, and the user's device displays this information as notifications or feeds.

[0240] Input: Final description

[0241] Output: Notifications or feeds delivered to other users

[0242] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0243] The system of the present invention is composed of a user terminal, a server, a generation AI, and an emotion engine. Below, we will provide a detailed explanation of how each element operates and how the entire system functions.

[0244] Operation on the user device

[0245] Enter and submit information

[0246] When a user is at an event or disaster site, they open a dedicated application. They write and attach information about the situation at the site in the form of text, images, or videos, and then press the send button. For example, they can post content such as "Band X is currently performing on Stage B, and a large audience is gathering." Location information and a timestamp are also automatically added from the user's device.

[0247] Processing on the server

[0248] Receiving and filtering information

[0249] The server receives the information sent from the user's device. The received content includes the posted text, images, videos, location information, and timestamps. After receiving the information, the server filters it using a specific algorithm to eliminate unnecessary or irrelevant information. The relevant information is then passed on to the next processing step.

[0250] Information clustering and keyword extraction

[0251] The filtered information is analyzed by a generation AI in the server, which clusters the filtered information into related groups (e.g., all information related to Stage B into one cluster).The generation AI then extracts important keywords from each cluster and generates a concise description.

[0252] Processing with the Emotion Engine

[0253] Emotion recognition

[0254] The emotion engine in the server analyzes information (text, images, videos, etc.) sent from the user's device and recognizes the user's emotions. For example, it determines whether the user is excited, confused, or relieved based on the wording in the text and facial expressions in the images.

[0255] Reflecting emotion in descriptions

[0256] Based on the user's emotional information recognized by the emotion engine, the generation AI reflects emotional elements in the explanation, adding emotional nuance to the explanation, making it possible to provide information that is easier to understand.

[0257] Generate and submit follow-up questions

[0258] Generate follow-up questions

[0259] The server then carefully examines the initially generated description to identify any missing information or ambiguities. Taking into account the user's emotions as recognized by the emotion engine, follow-up questions are automatically generated to confirm any additional information needed. For example, if the user is confused, a considerate question such as "Could you please explain the specific situation in more detail?" is generated.

[0260] Submit and answer questions

[0261] The server generates a follow-up question and sends it to the user, who then provides additional information in the form of text, images, etc. The answer is then sent back to the server.

[0262] Final information generation and distribution

[0263] Re-aggregating information and regenerating descriptions

[0264] The server receives additional information from the user and stores it in a database. The AI ​​then regenerates the final description based on the newly added information. For example, a specific description such as "Currently, band X is performing on stage B, and it's crowded. Excitement is building on the other stages as well."

[0265] Final information distribution

[0266] The server then distributes the resulting description to other users, who then display the information as notifications or feeds.

[0267] Specific examples

[0268] In the case of a music festival

[0269] 1. User A posts at a music festival, "Band X is performing on Stage B."

[0270] 2. The server receives this post and clusters it with other related posts.

[0271] 3. The AI ​​generates a description such as "Currently, band X is performing on stage B, so it is crowded."

[0272] 4. The server's emotion engine recognizes User A's excitement and reflects it in the description.

[0273] 5. User A is asked a further question and answers, "Band △△ is preparing for Stage C."

[0274] 6. The final explanation will be posted as follows: "Currently, band X is performing on Stage B, causing congestion. Band △△ is preparing on Stage C."

[0275] In case of disaster

[0276] 1. User B posts, "The roads are cracked due to the earthquake."

[0277] 2. The server receives this post and parses it along with other related posts.

[0278] 3. The AI ​​generates the following explanation: "The roads are cracked due to the earthquake. Aftershocks are continuing."

[0279] 4. The server's emotion engine recognizes User B's confusion and reflects it in the explanation.

[0280] 5. The server asks User B, "Where is the evacuation shelter?"

[0281] 6. User B answers, "The evacuation site is a nearby park."

[0282] 7. Finally, the following message is displayed: "Roads are cracked due to the earthquake. Aftershocks are continuing. Evacuation shelters are available at nearby parks."

[0283] The system of the present invention allows users to obtain fast and reliable information in real time and provides feedback that takes into account the user's emotions, thereby realizing a situation in which users can act with peace of mind during events or disasters.

[0284] The processing flow will be explained below.

[0285] Step 1:

[0286] The user opens the dedicated application, writes and attaches the situation at the site in the form of text, images, videos, etc., and presses the send button. Location information and a timestamp are automatically added from the user's device.

[0287] Step 2:

[0288] The server receives the information sent from the user terminal, including the posted text, images, videos, location information, and timestamps.

[0289] Step 3:

[0290] The server uses an algorithm to filter the information it receives, eliminating unnecessary or irrelevant information, and stores the filtered information in a database for analysis.

[0291] Step 4:

[0292] The filtered information is then analyzed by a generation AI on the server, which clusters highly relevant information and groups together information related to the same topic.

[0293] Step 5:

[0294] From the clustered information, the generative AI extracts important keywords, and generates an initial, concise description based on the extracted keywords.

[0295] Step 6:

[0296] The emotion engine analyzes posted text, images, and videos to recognize the user's emotions, such as whether they are excited, confused, or relieved.

[0297] Step 7:

[0298] Based on the user's recognized emotions, the generative AI reflects emotional elements in the description, adding emotional nuance to the description and making it easier to understand.

[0299] Step 8:

[0300] The server examines the generated explanation to identify missing or ambiguous information. The emotion engine takes the user's emotions into account and automatically generates follow-up questions to confirm any additional information needed.

[0301] Step 9:

[0302] The server generates and sends follow-up questions to the user, such as "What's the status of the other stages?" or "Could you please elaborate on the specific situation?"

[0303] Step 10:

[0304] The user terminal receives the additional question, and the user provides an answer (text, image, etc.) to the question. The user presses the send button to send the answer to the server.

[0305] Step 11:

[0306] The server receives additional information from the user and passes it to the generation AI for re-analysis, which regenerates the final description.

[0307] Step 12:

[0308] The server then distributes the resulting description to other users, who then display it in real time as notifications or feeds on their devices, allowing users to obtain fast and accurate information in real time.

[0309] This completes the system that recognizes the user's emotions and provides appropriate information.

[0310] Example 2

[0311] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0312] In modern society, there is a demand for fast and accurate information provision during events and disasters. However, providing information that takes into account the reliability, relevance, and user emotions presents challenges. Specifically, there is a need to quickly extract necessary information from a large amount of mixed information and provide feedback that takes into account the user's emotions.

[0313] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for a user to input and transmit information in the form of text, image, or video via a user terminal, means for automatically adding location information and a timestamp when information is transmitted from the user terminal, means for the server to receive information transmitted from the user terminal, means for filtering the received information using a specific algorithm and eliminating unnecessary information, means for a generation AI to analyze the filtered information and cluster related information, means for the generation AI to extract keywords from the clustered information and generate a concise explanation, means for an emotion engine in the server to recognize the user's emotions, means for the generation AI to reflect emotional elements in the explanation based on the emotion information, means for identifying deficiencies in the generated explanation and automatically generating additional questions to confirm additional information, means for the server to transmit the additional questions to the user terminal, means for a user to provide additional information and the server to receive it, means for regenerating a final explanation based on the additional information received by the server, and means for distributing the final explanation to other users. This will enable users to obtain fast and reliable information in real time during events or disasters, and also provide emotionally sensitive feedback.

[0314] "User terminal" refers to a device that a user uses to input and send information such as text, images, and videos, and includes smartphones, tablets, and computers.

[0315] "Location information" refers to geographic coordinate information obtained using the GPS function of the user's device, and indicates the specific location from which the user sent the information.

[0316] A "timestamp" is information that records the exact date and time when a user sends information, and is an important element when organizing and analyzing information later.

[0317] A "server" refers to a system that receives information sent from user terminals and performs processes such as analysis, filtering, clustering, and regeneration, and operates in cooperation with a large number of devices.

[0318] "Filtering" refers to the process of removing unnecessary or irrelevant information from received information and selecting only highly relevant information.

[0319] An "algorithm" refers to a set of computational or processing steps for solving a specific problem, and is a technique used for filtering and clustering information.

[0320] "Generative AI" refers to an artificial intelligence system that analyzes large amounts of data, extracts important keywords, and generates explanatory text.

[0321] "Clustering" refers to the process of grouping related information, and serves to bring together information that belongs to the same theme or category.

[0322] "Keyword extraction" refers to the process of extracting important words and phrases from clustered information, and is a fundamental step in generating concise descriptions.

[0323] An "emotion engine" is a system that analyzes information such as text, images, and videos sent by users and recognizes and judges their emotions.

[0324] A "concise explanation" is a summary sentence created by the generative AI based on clustered information and keywords, and provides information to users in a clear and easy-to-understand format.

[0325] The "additional question" refers to a question that is automatically generated to identify deficiencies in the generated explanation and to ask the user for additional information.

[0326] The "final description" refers to the complete information provided by the AI ​​after incorporating additional information from the user.

[0327] "Distribution" refers to the process of providing the final generated description to other users as a notification or feed, a fast and effective way to communicate information.

[0328] MODE FOR CARRYING OUT THE INVENTION

[0329] The system of the present invention is composed of a user terminal, a server, a generation AI, and an emotion engine. The operation of each element of the system and a specific embodiment of the entire system will be described below.

[0330] Operation on the user device

[0331] First, the user opens a dedicated application. The user writes and attaches information about the situation at the scene in the form of text, images, videos, etc. For example, if the user is at a music festival, they can post something like, "Band X is currently performing on Stage B, and a large audience is gathering." The user's device automatically adds location information and a timestamp to the entered data and sends it to the server.

[0332] Processing on the server

[0333] The server receives information sent from the user's device. The received content includes posted text, images, videos, location information, and timestamps. For accurate processing, the server filters the information using specific algorithms to eliminate unnecessary information. Next, the server's generation AI analyzes the filtered information and clusters related information. From the clustered information, the generation AI extracts important keywords and generates a concise description.

[0334] Processing with the Emotion Engine

[0335] The emotion engine in the server analyzes the information sent from the user's device and recognizes the user's emotions. Specifically, it determines the user's emotions based on the content of the text, facial expressions in images, and audio in videos. Based on this, the generation AI reflects emotional elements in the explanatory text. This makes it possible to provide information that is easier to understand.

[0336] Generate and submit follow-up questions

[0337] The generated explanation is then scrutinized by the server to identify any missing information or ambiguous parts. Taking into account the user's emotions recognized by the emotion engine, follow-up questions are automatically generated to confirm additional information. For example, if the user is confused, a considerate question such as "Could you please explain the specific situation in more detail?" is generated. The server then sends the generated follow-up question to the user's device. The user then provides additional information in response to the question, such as text or images, and sends it back to the server.

[0338] Final information generation and distribution

[0339] The server receives additional information from the user and stores it in a database. The generation AI regenerates the final description based on the newly added information. For example, a specific description such as "Currently, band X is performing on stage B, causing a crowd. Excitement is building on other stages as well" is generated. The server then distributes the final description to other users. The distributed information is displayed on the user's device as notifications or feeds.

[0340] Examples and prompts

[0341] Examples of music festivals

[0342] 1. User A posts at a music festival, "Band X is performing on Stage B."

[0343] 2. The server receives this post and clusters it with other related posts.

[0344] 3. The AI ​​generates a description such as "Currently, band X is performing on stage B, so it is crowded."

[0345] 4. The server's emotion engine recognizes User A's excitement and reflects it in the description.

[0346] 5. User A is asked a further question and answers, "Band △△ is preparing for Stage C."

[0347] 6. The final explanation will be posted as follows: "Currently, band X is performing on Stage B, causing congestion. Band △△ is preparing on Stage C."

[0348] Prompt Sentence Examples

[0349] "What kind of description should be generated if the user is excited at a music festival?"

[0350] Specific examples of disasters

[0351] 1. User B posts, "The roads are cracked due to the earthquake."

[0352] 2. The server receives this post and parses it along with other related posts.

[0353] 3. The AI ​​generates the following explanation: "The roads are cracked due to the earthquake. Aftershocks are continuing."

[0354] 4. The server's emotion engine recognizes User B's confusion and reflects it in the explanation.

[0355] 5. The server asks User B, "Where is the evacuation shelter?"

[0356] 6. User B answers, "The evacuation site is a nearby park."

[0357] 7. Finally, the following message is displayed: "Roads are cracked due to the earthquake. Aftershocks are continuing. Evacuation shelters are available at nearby parks."

[0358] Prompt Sentence Examples

[0359] "What kind of explanatory text should be generated when a user is confused during a disaster?"

[0360] The system of the present invention allows users to obtain fast and reliable information in real time and receive feedback that takes into account their emotions, thereby realizing a situation in which users can act with peace of mind during events or disasters.

[0361] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0362] Step 1: User opens dedicated application

[0363] How it works: The user launches a dedicated application on their device.

[0364] Input: User operation (application startup)

[0365] Output: The application starts and the input screen is displayed.

[0366] Step 2: User inputs and attaches the situation on site and sends it

[0367] How it works: The user enters text into the text box, attaches images and videos, for example, "Band X is currently performing on Stage B," and then presses the send button.

[0368] Input: Text data, image files, video files

[0369] Output: An information packet with automatically added location and timestamp is generated and sent to the server.

[0370] Step 3: The server receives the information

[0371] Operation: The server receives an information packet sent from the user terminal.

[0372] Input: Information packets (text, images, videos, location information, timestamps)

[0373] Output: The received information is stored in a database.

[0374] Step 4: The server filters the information

[0375] How it works: The server uses specific algorithms to filter out unwanted information from the information it receives, for example filtering out spam messages and irrelevant images.

[0376] Input: Incoming data (text, image, video, location, timestamp)

[0377] Output: Filtered and relevant information is extracted.

[0378] Step 5: Server clustering information

[0379] How it works: A generative AI in the server analyzes the filtered information and groups highly relevant information together, for example, all posts about Stage B into one cluster.

[0380] Input: Filtered information

[0381] Output: Clustered information (information within a cluster is integrated)

[0382] Step 6: Generative AI extracts keywords and generates descriptions

[0383] How it works: The generative AI extracts important keywords from the clustered information and generates a concise description based on them. For example, it might generate a description like, "Currently, band X is performing on Stage B, so it may be crowded."

[0384] Input: Clustered information

[0385] Output: Description

[0386] Step 7: The emotion engine recognizes the user's emotions

[0387] How it works: The emotion engine in the server analyzes the information sent by the user and recognizes the user's emotions, for example, determining excitement or confusion from the wording in the text and facial expressions in the images.

[0388] Input: Information sent (text, images, videos)

[0389] Output: User's emotional information

[0390] Step 8: Reflect emotions in your description

[0391] How it works: Based on the emotional information recognized by the emotion engine, the generation AI reflects emotional elements in the description. For example, it adds the expression "There is a large audience and the atmosphere is lively."

[0392] Input: Emotion information, generated description

[0393] Output: Final description reflecting the sentiment

[0394] Step 9: The server generates and sends a challenge requesting additional information.

[0395] How it works: The server examines the generated explanation, identifies missing information, and automatically generates follow-up questions. For example, it generates a question like, "Could you please tell me more about your specific situation?" and sends it to the user's device.

[0396] Input: Final description

[0397] Output: Additional questions

[0398] Step 10: User provides additional information and submits

[0399] How it works: The user responds to a follow-up question from the server by providing additional information, such as text or images, and submitting it.

[0400] Input: Additional information (text, image)

[0401] Output: Additional information is sent to the server.

[0402] Step 11: Server receives additional information and re-aggregates

[0403] How it works: The server receives additional information sent by the user and re-aggregates it with existing data.

[0404] Input: Additional Information

[0405] Output: Complete information aggregated in a database

[0406] Step 12: The generation AI regenerates the final description

[0407] How it works: The AI ​​then regenerates the final description based on the re-aggregated information. For example, it might generate a specific description like, "Currently, band X is performing on stage B, causing congestion. Band Y is preparing on stage C."

[0408] Input: Re-aggregated information

[0409] Output: Final description

[0410] Step 13: The server delivers the final description

[0411] How it works: The server distributes the final description to other users, who then display it as notifications or feeds on their devices.

[0412] Input: Final description

[0413] Output: Notifications and feeds delivered to user devices

[0414] This allows users to obtain fast and reliable information in real time, and provides emotionally sensitive feedback. This system creates an environment where users can act with confidence during events and disasters.

[0415] (Application example 2)

[0416] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0417] The rapid sharing and efficient management of information within a logistics center is extremely important for improving work efficiency and reducing errors. However, there are often delays in collecting information from the field and providing feedback based on that information, which can result in work delays and incorrect deliveries. Furthermore, there is a need to provide optimal feedback based on the emotions and urgency of field workers, but current systems have difficulty in this regard. To solve these issues, a system is needed that quickly and accurately collects field information and provides feedback that takes into account the emotions of workers.

[0418] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0419] In this invention, the server includes means for receiving information transmitted from a user terminal, means for filtering the received information to eliminate unnecessary information, means for analyzing the filtered information and clustering related information, means for extracting keywords from the clustered information and generating a concise explanation, means for analyzing the emotional state of the user using an emotion engine and reflecting the emotion in the explanation, means for identifying deficiencies in the generated explanation and transmitting additional information to the user, means for receiving the additional information and regenerating a final explanation, and means for distributing the final explanation to other users. This improves the efficiency of information sharing and management within the logistics center and enables prompt and appropriate feedback that takes into account the emotions of field workers.

[0420] A "user terminal" is a device used by workers in a logistics center to input and transmit information.

[0421] A "server" is a device that receives information sent from a user terminal, and performs filtering, analysis, clustering, description generation, sentiment analysis, sending additional information, and regenerating final information.

[0422] "Filtering" refers to the process of filtering out unnecessary information from received information and extracting only the necessary information.

[0423] "Clustering" refers to the process of grouping filtered information based on relevance.

[0424] "Keyword extraction" is the process of selecting important elements from clustered information.

[0425] "Description generation" is the process of creating concise, easy-to-understand text based on the important elements obtained through keyword extraction.

[0426] An "emotion engine" is a system that analyzes information input by a user and recognizes the user's emotional state from that information.

[0427] "Sending additional information" refers to an operation of sending a question or request to the user to fill in any gaps in the generated explanation.

[0428] "Receiving additional information" refers to an operation of receiving supplemental information sent by a user.

[0429] "Regeneration of final information" is the process of updating the description again based on additional information to make it more accurate.

[0430] "Delivery of final information" refers to an operation of notifying other users of the finally generated explanation.

[0431] "Emotion analysis" refers to the process of analyzing a user's emotional state through an emotion engine.

[0432] System Configuration

[0433] This invention is a system consisting of a user terminal, a server, a generative AI, and an emotion engine. The user terminal is used by workers in a logistics center and is responsible for inputting on-site information and sending it to the server. The server is a device that receives, processes, analyzes, recognizes emotions, and finally regenerates and distributes explanatory text.

[0434] Operation on the user device

[0435] On the user device, workers use a dedicated smartphone application to input on-site information. Input can be in a variety of formats, including text, images, and videos, and there is a function to input and send specific information, such as "the package label cannot be found in Section B." Location information and timestamps are also automatically added from the user device. This makes it possible to clearly identify when and where each piece of information occurred.

[0436] Processing on the server

[0437] The server receives information sent from the user terminal. The received information is first processed by a filtering algorithm to remove unnecessary or irrelevant information. Next, highly relevant information is clustered and managed in a consolidated manner.

[0438] Generative AI analyzes the clustered information, extracts important keywords, and generates concise descriptions that summarize the filtered information and convert it into an easy-to-understand format.

[0439] Processing with the Emotion Engine

[0440] The emotion engine in the server analyzes the information sent from the user's device and recognizes the worker's emotional state. For example, emotions such as "confusion," "excitement," and "relief" are determined through text and image analysis. This adds emotional elements to the generated explanation, enabling the system to provide more considerate feedback to the user.

[0441] Generate and submit follow-up questions

[0442] The server then carefully examines the initially generated description to identify any missing or ambiguous information. In such cases, the AI ​​takes into account the user's emotions as recognized by the emotion engine and automatically generates questions to request additional information. For example, for a confused user, it generates a thoughtful question such as, "Could you please explain the specific situation in more detail?"

[0443] Final information generation and distribution

[0444] The server receives additional information from the user and stores it in a database. The generation AI regenerates the final explanation based on the newly added information. For example, a specific explanation such as "The package label cannot be confirmed in section B. Upon checking, we found that the label was missing, so we attached a new label" is generated. The final explanation is notified to other workers and managers, enabling information sharing in real time.

[0445] Specific examples

[0446] For example:

[0447] Example 1: Information sharing in a logistics center

[0448] 1. Worker A types "I can't find the package label in Section B" into his smartphone and sends it.

[0449] 2. The server receives this information and performs filtering and clustering.

[0450] 3. The AI ​​generates the explanation "The package label cannot be confirmed in section B."

[0451] 4. The emotion engine recognizes worker A's confusion and reflects it in the explanation.

[0452] 5. A follow-up question is generated and sent to worker A: "Could you please tell me more about the specific situation?"

[0453] 6. Worker A responds, "I checked and found that the label was missing, so I attached a new label."

[0454] 7. The final explanation is sent to other workers: "Section B could not find the package label. Upon checking, we found that the label was missing, so we attached a new label."

[0455] Prompt Sentence Examples

[0456] "User Input: Package label not found in Section B."

[0457] "Generative AI model prompt: Please describe the situation on the ground."

[0458] "Sentiment analysis: puzzled"

[0459] "Final description: Problem with section B. Details: Unable to locate package label. Emotion: Confused."

[0460] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0461] Step 1:

[0462] Entering information on the user's device

[0463] A worker with a user device reports a specific problem or situation at the logistics center. The worker opens a dedicated smartphone application and inputs the situation in the form of text, images, or videos, such as "The package label cannot be found in Section B." The user device automatically adds location information and a timestamp.

[0464] Input: On-site information (text, images, videos), location information, timestamp

[0465] Output: Structured information data

[0466] Step 2:

[0467] Receiving and filtering information

[0468] The server receives the information sent from the user's device and filters it using specific algorithms to remove unnecessary or irrelevant information, such as typos and meaningless data from the text.

[0469] Input: Structured information data

[0470] Output: Filtered information data

[0471] Step 3:

[0472] Information Clustering

[0473] The server analyzes the filtered information and clusters related information, for example, all "questions related to Section B" into one cluster, thereby aggregating information for specific sections or situations.

[0474] Input: Filtered information data

[0475] Output: Clustered information data

[0476] Step 4:

[0477] Keyword extraction and description generation

[0478] The server's generation AI extracts important keywords from the clustered information and generates a concise explanation, such as "The luggage label cannot be found in section B."

[0479] Input: Clustered information data

[0480] Output: Generated description

[0481] Step 5:

[0482] Emotion recognition

[0483] The server uses an emotion engine to analyze information sent from the user's device and recognize the worker's emotional state. For example, it can determine whether the worker is in a "confused" state from text and images.

[0484] Input: Generated description, information from user terminal

[0485] Output: Recognized emotion data

[0486] Step 6:

[0487] Generate and submit follow-up questions

[0488] If the server determines that the generated description is insufficient, the AI ​​generates additional questions based on the emotion recognition results, such as "Could you please tell me more about the specific situation?"

[0489] Input: Generated description, recognized emotion data

[0490] Output: Additional questions

[0491] Step 7:

[0492] Receiving additional information from the user

[0493] The user terminal receives the query from the server, and the worker enters specific additional information and resubmits it. For example, the worker returns information such as, "After checking, I found that a label was missing, so I attached a new label."

[0494] Input: Additional Question

[0495] Output: Additional information data

[0496] Step 8:

[0497] Regenerate the final information

[0498] The server re-analyzes the generated AI based on the additional information and re-generates the final description, for example, "The package label cannot be found in section B. Upon checking, the label was found to be missing, so a new label was attached."

[0499] Input: Additional information data

[0500] Output: Final description

[0501] Step 9:

[0502] Final information distribution

[0503] The server notifies other users of the final generated explanation, for example, by distributing it in real time to other workers or administrators.

[0504] Input: Final description

[0505] Output: Notification description

[0506] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0507] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0508] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0509] [Second embodiment]

[0510] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0511] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0512] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0513] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0514] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0515] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0516] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0517] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0518] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0519] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0520] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0521] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0522] The system of the present invention is composed of a user terminal, a server, and a generation AI. Below, we will provide a detailed explanation of how each element operates and how the entire system functions.

[0523] Operation on the user device

[0524] Enter and submit information

[0525] When a user is at an event or disaster site, they open a dedicated application. They write and attach information about the situation at the site in the form of text, images, or videos, and then press the send button. For example, they could post something like, "Band X is currently performing on Stage B, and a large audience is gathering." At this time, location information and a timestamp are automatically added from the user's device.

[0526] Processing on the server

[0527] Receiving and filtering information

[0528] The server receives information sent from the user's device. The received information includes the content of the post, location information, timestamp, etc. After receiving the information, the server filters it using a specific algorithm to eliminate unnecessary or irrelevant information. The relevant information is then passed on to the next processing step.

[0529] Information clustering and keyword extraction

[0530] The filtered information is analyzed by a generation AI in the server, which clusters the filtered information into related groups (e.g., all information related to Stage B into one cluster).The generation AI then extracts important keywords from each cluster and generates a concise description.

[0531] Examining the generated explanation and generating follow-up questions

[0532] The server then carefully examines the generated description to identify any missing information or ambiguities. For example, if the generated description is "Stage B is currently crowded because band X is playing," the server will automatically generate additional questions about the situation on other stages and the number of audience members.

[0533] Asking the user additional questions and completing information

[0534] Submit and answer additional questions

[0535] The server generates a follow-up question and sends it to the user. For example, a question like "What is the status of the other stages?" is sent to the user. The user responds to the question by providing additional information in the form of text or images. This answer is then sent back to the server.

[0536] Final information generation and distribution

[0537] Re-aggregating information and regenerating descriptions

[0538] The server receives additional information from the user and stores it in a database. The generation AI then regenerates the final description based on the newly added information. For example, a specific description such as "Currently, band X is performing on stage B, causing congestion. Band Y is preparing on stage C." is generated.

[0539] Final information distribution

[0540] The server then distributes the resulting descriptions to other users in real time, who then display the information as notifications or feeds.

[0541] Specific examples

[0542] Music festival scenario

[0543] 1. User A posts at a music festival, "Band X is performing on Stage B."

[0544] 2. The server receives this post and clusters it with other related posts.

[0545] 3. The AI ​​generates a description like "Stage B is currently crowded because band X is performing," but it lacks information about the other stages.

[0546] 4. The server sends a follow-up question to User A: "What's the status of the other stages?"

[0547] 5. User A answers, "Band △△ is preparing for stage C."

[0548] 6. The final description will be "Currently, stage B is crowded as band X is performing. band YY is preparing on stage C."

[0549] Disaster scenarios

[0550] 1. User B posts, "The roads are cracked due to the earthquake."

[0551] 2. The server receives this post and parses it along with other related posts.

[0552] 3. The AI ​​generates the description "Roads are cracked due to the earthquake. Aftershocks are continuing," but it lacks information about evacuation sites.

[0553] 4. The server asks User B an additional question: "Where is the evacuation site?"

[0554] 5. User B answers, "The evacuation site is a nearby park."

[0555] 6. Finally, the following message is displayed: "Roads are cracked due to the earthquake. Aftershocks are continuing. Evacuation shelter is in a nearby park."

[0556] The system of the present invention allows users to obtain fast and reliable information in real time, enabling accurate situational understanding even during events or disasters.

[0557] The processing flow will be explained below.

[0558] Step 1:

[0559] The user opens the dedicated application, writes and attaches the situation at the site in the form of text, images, videos, etc., and presses the send button. Location information and a timestamp are automatically added from the user's device.

[0560] Step 2:

[0561] The server receives the information sent from the user terminal, including the posted text, images, videos, location information, and timestamps.

[0562] Step 3:

[0563] The server filters the received information using specific algorithms to remove unnecessary or irrelevant information, and stores the filtered information for analysis.

[0564] Step 4:

[0565] Based on the filtered information, the generative AI clusters this information, specifically grouping highly related information together.

[0566] Step 5:

[0567] From the clustered information, the generative AI extracts important keywords, and generates an initial, concise description based on the extracted keywords.

[0568] Step 6:

[0569] The server reviews the generated description to identify any missing or ambiguous information, confirms the additional information needed, and proceeds to the next step.

[0570] Step 7:

[0571] The server automatically generates additional questions based on the missing information and sends the generated questions to the corresponding user terminal.

[0572] Step 8:

[0573] The user terminal receives the additional question, and the user provides an answer (text, image, etc.) to the question. The user presses the send button to send the answer to the server.

[0574] Step 9:

[0575] The server receives additional information from the user and passes it back to the AI ​​for re-analysis, which then generates the final description.

[0576] Step 10:

[0577] The server then distributes the final description to other users, who then display it in real time as notifications or feeds on their devices.

[0578] This allows users to obtain fast and accurate information in real time.

[0579] Example 1

[0580] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0581] During modern events and disasters, it is important to quickly and accurately grasp the situation at the scene, but manually collecting and organizing information takes time and effort. It is also difficult to ensure the reliability of the information and real-time availability of the information. Furthermore, when efficiently processing multi-format data (text, images, and video) and converting it into easy-to-understand descriptions, additional information from users is often not reflected in a timely manner. To address these issues, effective information reception, filtering, clustering, keyword extraction, and improved accuracy of automatically generated descriptions are required.

[0582] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0583] In this invention, the server includes means for receiving multi-format data transmitted from a user terminal, means for initially processing the received data and eliminating irrelevant data, means for clustering the filtered data and organizing it into related information groups, means for extracting keywords from the clustered information groups and generating an explanatory text, means for identifying missing information in the generated explanatory text and requesting additional information from the user, means for receiving the additional information from the user and regenerating a final explanatory text, and means for distributing the final explanatory text to other users in real time, thereby enabling efficient processing of multi-format data and providing reliable information in real time.

[0584] A "user terminal" is a computing device that a user uses to input and transmit information, and includes smartphones, tablets, laptops, and the like.

[0585] "Multi-format data" refers to digital data in different formats, such as text, images, and video.

[0586] A "server" is a computer system that receives, processes, stores, and distributes data sent from user terminals over a network.

[0587] "Initial processing" refers to processing to analyze received data and eliminate signal noise and unnecessary information.

[0588] "Filtering" refers to the process of removing unnecessary information from received data and extracting highly relevant information.

[0589] "Clustering" is a technical method for grouping data that have similar characteristics.

[0590] An "information group" is a collection of a series of clustered related information.

[0591] "Keyword extraction" is the process of identifying and extracting important words and phrases from text data.

[0592] "Explanation" refers to text information generated in a format that is easy for users to understand.

[0593] A "generative AI model" is an algorithm or model that uses artificial intelligence to analyze data, extract keywords, and generate descriptions.

[0594] "Additional information" refers to information requested from the user to reacquire data that was missing at the time of initial information collection.

[0595] "Real-time delivery" is the process of providing the results to other users immediately at the moment the data is collected and analyzed.

[0596] The system of the present invention is composed of a user terminal, a server, and a generative AI model. Below, we will explain in detail how each element operates and how the entire system functions.

[0597] Hardware and Software Configuration

[0598] User device:

[0599] A user device is a computing device used by a user, and includes smartphones, tablets, laptops, etc. Using these devices, users open a dedicated application and enter information about an event or disaster site in the form of text, images, and videos. For example, a user might enter text such as "Band X is currently performing on Stage B," attach a photo, and press the send button. The user device automatically adds location information and a timestamp before sending the information.

[0600] server:

[0601] The server is a computer system that receives and processes multi-format data sent from user devices via a network. The received data is stored in a real-time database, and irrelevant and inconsistent data is eliminated as an initial process. The server then filters the data using a specific algorithm to extract only highly relevant data. The filtered data is then clustered using a generative AI model and organized into related information groups. Machine learning techniques are used for this clustering process. The generative AI model then extracts keywords from each cluster and generates a concise explanation to be communicated to the user.

[0602] Generative AI models:

[0603] The generative AI model is an algorithm that extracts keywords from the clustered information and generates explanatory text. If the generated explanation is incomplete, the server identifies this and generates and sends a prompt to the user requesting additional information.

[0604] Specific examples

[0605] Music festival scenario

[0606] 1. User A is at a music festival and types in the text "XX band is performing on Stage B," attaches a photo of the stage, and presses the send button.

[0607] 2. The server receives this information and stores the data, including location and timestamp, in a real-time database.

[0608] 3. A filtering algorithm is applied to remove unnecessary data.

[0609] 4. The generative AI model clusters related information and combines information related to "Stage B" into a single cluster.

[0610] 5. The generative AI model extracts keywords from the cluster and generates a description such as, "Stage B is crowded because band X is performing."

[0611] 6. The server generates a follow-up question: "What is the status of the other stages?" and notifies User A.

[0612] 7. User A replies, "Band △△ is being prepared on Stage C," and the information is sent again.

[0613] 8. The server receives the additional information, and the generative AI model regenerates the final description: "Currently, stage B is crowded because band X is performing. On stage C, band YY is preparing."

[0614] 9. The server distributes the final information to other users in real time.

[0615] Disaster scenarios

[0616] 1. User B posts, "The roads are cracked due to the earthquake," attaches a photo of the scene, and presses the send button.

[0617] 2. The server receives this information and stores it in a real-time database.

[0618] 3. A filtering algorithm is applied to extract only the required data.

[0619] 4. The generative AI model clusters related information and combines information about "earthquakes" into one cluster.

[0620] 5. The generative AI model extracts keywords from the cluster and generates a description such as, "The roads are cracked due to the earthquake. Aftershocks are continuing."

[0621] 6. The server generates an additional question, "Where is the evacuation site?" and notifies User B.

[0622] 7. User B replies, "The evacuation site is a nearby park," and the information is sent again.

[0623] 8. The server receives the additional information, and the generative AI model regenerates the final description: "Roads are cracked due to the earthquake. Aftershocks are continuing. Evacuation shelters are in nearby parks."

[0624] 9. The server distributes the final information to other users in real time.

[0625] This allows users to obtain fast and reliable information in real time, enabling accurate situational understanding even during events or disasters.

[0626] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0627] Step 1:

[0628] The user opens a dedicated application. The user enters information about the event or disaster site in text, image, and video format, and presses the "Send" button. At this time, the user's device automatically adds location information and a timestamp. A specific example of how this works is that the user enters "Band X is currently performing on Stage B," attaches a photo of the scene, and presses the "Send" button. The input data consists of text, images, location information, and a timestamp, and is then sent to the server.

[0629] Step 2:

[0630] The server receives multi-format data sent from user devices. The server performs initial processing of the received data and applies filtering algorithms to remove irrelevant or inconsistent data. The input data is all the information sent by the user, and the output data is clean data with unnecessary data removed. Specific operations include removing spam messages and checking format consistency.

[0631] Step 3:

[0632] The server clusters the filtered data. Using a generative AI model, related information is grouped and organized into related information clusters. The input data is the filtered data, and the output data is the clustered information clusters. Specifically, multiple posts about the same event are grouped into one cluster.

[0633] Step 4:

[0634] The server uses a generative AI model to extract keywords from the clustered information and generate a description. The input data is the clustered information, and the output data is the generated description. For example, a description such as "Stage B is crowded because band X is performing" is automatically generated.

[0635] Step 5:

[0636] The server examines the generated description and identifies any missing or ambiguous information. Based on this, the server generates a prompt to request additional information from the user. The input data is the generated description, and the output data is a prompt for additional information. Specifically, a question such as "What is the status of the other stages?" is generated.

[0637] Step 6:

[0638] The server notifies the user terminal of the generated prompt text. The user receives the notification and inputs additional information. This additional information is also sent in the form of text, images, or video. The input data is the additional information sent by the user, and the output data is additional data including the additional information. A specific operation is for the user to reply, "△△ band is being prepared on stage C," and send it.

[0639] Step 7:

[0640] The server receives additional information from the user and regenerates the final description using the generative AI model. The input data is the additional information from the user, and the output data is the regenerated final description. A specific example of how this works is that the final description might read, "Currently, band X is performing on stage B, causing congestion. Band Y is preparing on stage C."

[0641] Step 8:

[0642] The server distributes the generated final description to other users in real time. The user's device displays the final information in the form of a notification or feed. The input data is the final description, and the output data is the distributed notification. Specifically, the smartphone notification screen will display "Currently, band X is performing on stage B, causing congestion. Band Y is preparing on stage C."

[0643] This allows the system to efficiently process information from users and provide other users with reliable information in real time.

[0644] (Application example 1)

[0645] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0646] In modern society, it is extremely important to quickly and accurately collect information about abnormal situations and emergencies and notify relevant parties. However, current information collection systems often contain a lot of unnecessary information and lack real-time capabilities, making it difficult to respond quickly. Information is also often insufficient or ambiguous, making it difficult to collect additional information. This can hinder appropriate responses in emergencies. It is necessary to solve these problems and achieve efficient and rapid reporting and response to abnormal situations.

[0647] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0648] In this invention, the server includes: means for receiving information transmitted from a user terminal; means for filtering the received information to eliminate unnecessary information; means for analyzing the filtered information and clustering relevant information; means for extracting keywords from the clustered information and generating a concise explanation; means for identifying deficiencies in the generated explanation and transmitting additional information to the user; means for receiving the additional information and regenerating a final explanation; means for distributing the final explanation to other users; means for comprehensively analyzing the anomaly report information transmitted from the user and detecting special circumstances; and means for automatically generating follow-up questions based on the special circumstances and transmitting them to the user in real time. This enables the server to quickly and accurately collect and analyze anomaly information reported by users and efficiently supplement missing information. The generated information can then be instantly distributed to relevant parties, enabling a rapid response.

[0649] A "user terminal" is an electronic device operated by a user, and is a terminal device for inputting and transmitting information.

[0650] "Information" refers to all data sent from a user terminal, such as text, images, videos, location information, and timestamps.

[0651] "Means for receiving" refers to a mechanism for receiving and storing information sent from a user terminal.

[0652] A "filtering means" is a mechanism for filtering out unnecessary or irrelevant information from received information.

[0653] The "means for analyzing and clustering related information" is a method for analyzing the filtered information and grouping related information into one group.

[0654] "Means for extracting keywords and generating concise descriptions" is a technology for extracting important words and phrases from clustered information and creating concise sentences based on them.

[0655] The "means for identifying missing parts of the generated description and sending additional information to the user" is a mechanism for verifying the generated description, identifying missing information, and inquiring about that information from the user.

[0656] The "means for receiving additional information and regenerating the final description" is a method for creating a new, compiled final description based on the additional information provided by the user.

[0657] The "means for distributing the final description to other users" is a mechanism for transmitting the generated final description to other users in real time.

[0658] "Anomaly report information" refers to information reported by a user regarding an abnormality or emergency.

[0659] The "means for detecting special situations" is a method for analyzing abnormality report information and identifying abnormal situations based on specific conditions or patterns.

[0660] The "means for automatically generating follow-up questions and sending them to the user in real time" is a mechanism for generating necessary follow-up questions based on the detected abnormal situation and sending them to the user immediately.

[0661] The system of the present invention is composed of a user terminal, a server, and a generation AI. Below, we will provide a detailed explanation of how each element operates and how the entire system functions.

[0662] Operation on the user device

[0663] Enter and submit information:

[0664] Users use a dedicated application installed on their smartphones or smart glasses. When they witness an abnormal or emergency situation, they record the situation on the scene in the form of text, images, or videos, and press the send button. At this time, location information and a timestamp are automatically added. For example, a user might post, "I spotted a suspicious person. I'm attaching a photo."

[0665] Processing on the server

[0666] Receiving and filtering information:

[0667] The server receives the information sent from the user's device and uses the server's computing resources (e.g., AWS, Google Cloud Platform) and a filtering algorithm to remove unnecessary data from the received information. For example, it performs processing such as "excluding information about suspicious individuals."

[0668] Information clustering and keyword extraction:

[0669] The filtered information is analyzed by a generative AI (e.g., a GPT model) on the server. The generative AI uses a clustering algorithm to group related information together. It then extracts important keywords and generates a concise description. For example, it creates a description such as, "A suspicious person has been spotted in a specific location. We would like to report the current situation."

[0670] Examining the generated explanation and generating follow-up questions:

[0671] The server then examines the generated description to identify any missing information or ambiguity. For example, if the generated description is "A suspicious person was spotted in a specific location," the server automatically generates follow-up questions such as "What is the situation in other areas?". The server uses a generative AI model and prompts to generate follow-up questions.

[0672] Asking the user additional questions and completing information

[0673] Submit and answer additional questions:

[0674] The server generates a follow-up question and sends it to the user. For example, the user might ask, "What is the situation in other areas?" The user then provides additional information in the form of text or images. This answer is then sent back to the server.

[0675] Final information generation and distribution

[0676] Re-aggregate information and regenerate descriptions:

[0677] The server receives additional information from the user and stores it in a database. The AI ​​then regenerates the final description based on the newly added information. For example, a specific description such as "A suspicious person was spotted in a specific location. Other areas are being investigated."

[0678] Final information distribution:

[0679] The server then distributes the resulting description to other users in real time, and the user's device displays this information as a notification or feed, enabling fast and accurate information sharing.

[0680] Examples of concrete examples and prompts

[0681] Examples:

[0682] 1. A user reports that they have spotted a suspicious person in a specific location.

[0683] 2. The server receives the reports and filters and clusters the relevant information.

[0684] 3. The AI ​​generates a description that says "A suspicious person was spotted in a specific location," but lacks information about other areas.

[0685] 4. The server sends a follow-up question to the user: "What's the situation in other areas?"

[0686] 5. The user replies, "There are no abnormalities in other areas."

[0687] 6. The final description will be "Suspicious person spotted in specific location. No abnormalities in other areas."

[0688] Example prompt sentence:

[0689] A user reports a suspicious person in a specific location. Please suggest the next course of action based on the current situation on the scene.

[0690] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0691] Step 1:

[0692] Entering and sending information on the user's device

[0693] When a user witnesses an abnormal or emergency situation, they input information using a dedicated application installed on their smartphone or smart glasses. The input information is recorded in text, image, or video format and sent to a server by pressing the send button. Location information and a timestamp are automatically added.

[0694] Input: Text, images, videos, with automatically added location and timestamps

[0695] Output: Anomaly information sent to the server (text, images, videos, location information, timestamp)

[0696] Step 2:

[0697] Receiving and filtering information on the server

[0698] The server receives the anomaly information sent from the user terminal, and then uses a filtering algorithm to filter out unnecessary or irrelevant information.

[0699] Input: Anomaly information submitted by the user (text, image, video, location, timestamp)

[0700] Output: Filtered and relevant anomaly information

[0701] Step 3:

[0702] Server-based information clustering and keyword extraction

[0703] The filtered information is analyzed using generative AI (e.g., GPT model) to cluster relevant information, and then important keywords are extracted from the clustered information to generate a concise description.

[0704] Input: Filtered anomaly information

[0705] Output: Generated description (e.g. "A suspicious person was spotted at a specific location")

[0706] Step 4:

[0707] The server examines the generated explanation and generates follow-up questions

[0708] The server then examines the generated explanation to identify any missing information or ambiguity. It then automatically generates follow-up questions using a generative AI model and creates prompts to notify the user.

[0709] Input: Generated description

[0710] Output: Prompt for follow-up questions (e.g., "What about other areas?")

[0711] Step 5:

[0712] Sending and answering follow-up questions to users

[0713] The server generates a follow-up question and sends it to the user, who then provides additional information in the form of text, images, etc. The answer is then sent back to the server.

[0714] Input: Prompt for follow-up questions, additional information from the user (text, images, etc.)

[0715] Output: Additional information sent to the server

[0716] Step 6:

[0717] Server re-aggregates information and regenerates descriptions

[0718] The server receives additional information from the user and stores it in a database. The AI ​​then regenerates the final description based on the newly added information.

[0719] Input: Additional information from the user

[0720] Output: Final description (e.g. "Suspicious individuals were spotted in specific locations. No anomalies found in other areas.")

[0721] Step 7:

[0722] Final information generation and distribution

[0723] The server then distributes the resulting description to other users in real time, and the user's device displays this information as notifications or feeds.

[0724] Input: Final description

[0725] Output: Notifications or feeds delivered to other users

[0726] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0727] The system of the present invention is composed of a user terminal, a server, a generation AI, and an emotion engine. Below, we will provide a detailed explanation of how each element operates and how the entire system functions.

[0728] Operation on the user device

[0729] Enter and submit information

[0730] When a user is at an event or disaster site, they open a dedicated application. They write and attach information about the situation at the site in the form of text, images, or videos, and then press the send button. For example, they can post content such as "Band X is currently performing on Stage B, and a large audience is gathering." Location information and a timestamp are also automatically added from the user's device.

[0731] Processing on the server

[0732] Receiving and filtering information

[0733] The server receives the information sent from the user's device. The received content includes the posted text, images, videos, location information, and timestamps. After receiving the information, the server filters it using a specific algorithm to eliminate unnecessary or irrelevant information. The relevant information is then passed on to the next processing step.

[0734] Information clustering and keyword extraction

[0735] The filtered information is analyzed by a generation AI in the server, which clusters the filtered information into related groups (e.g., all information related to Stage B into one cluster).The generation AI then extracts important keywords from each cluster and generates a concise description.

[0736] Processing with the Emotion Engine

[0737] Emotion recognition

[0738] The emotion engine in the server analyzes information (text, images, videos, etc.) sent from the user's device and recognizes the user's emotions. For example, it determines whether the user is excited, confused, or relieved based on the wording in the text and facial expressions in the images.

[0739] Reflecting emotion in descriptions

[0740] Based on the user's emotional information recognized by the emotion engine, the generation AI reflects emotional elements in the explanation, adding emotional nuance to the explanation, making it possible to provide information that is easier to understand.

[0741] Generate and submit follow-up questions

[0742] Generate follow-up questions

[0743] The server then carefully examines the initially generated description to identify any missing information or ambiguities. Taking into account the user's emotions as recognized by the emotion engine, follow-up questions are automatically generated to confirm any additional information needed. For example, if the user is confused, a considerate question such as "Could you please explain the specific situation in more detail?" is generated.

[0744] Submit and answer questions

[0745] The server generates a follow-up question and sends it to the user, who then provides additional information in the form of text, images, etc. The answer is then sent back to the server.

[0746] Final information generation and distribution

[0747] Re-aggregating information and regenerating descriptions

[0748] The server receives additional information from the user and stores it in a database. The AI ​​then regenerates the final description based on the newly added information. For example, a specific description such as "Currently, band X is performing on stage B, and it's crowded. Excitement is building on the other stages as well."

[0749] Final information distribution

[0750] The server then distributes the resulting description to other users, who then display the information as notifications or feeds.

[0751] Specific examples

[0752] In the case of a music festival

[0753] 1. User A posts at a music festival, "Band X is performing on Stage B."

[0754] 2. The server receives this post and clusters it with other related posts.

[0755] 3. The AI ​​generates a description such as "Currently, band X is performing on stage B, so it is crowded."

[0756] 4. The server's emotion engine recognizes User A's excitement and reflects it in the description.

[0757] 5. User A is asked a further question and answers, "Band △△ is preparing for Stage C."

[0758] 6. The final explanation will be posted as follows: "Currently, band X is performing on Stage B, causing congestion. Band △△ is preparing on Stage C."

[0759] In case of disaster

[0760] 1. User B posts, "The roads are cracked due to the earthquake."

[0761] 2. The server receives this post and parses it along with other related posts.

[0762] 3. The AI ​​generates the following explanation: "The roads are cracked due to the earthquake. Aftershocks are continuing."

[0763] 4. The server's emotion engine recognizes User B's confusion and reflects it in the explanation.

[0764] 5. The server asks User B, "Where is the evacuation shelter?"

[0765] 6. User B answers, "The evacuation site is a nearby park."

[0766] 7. Finally, the following message is displayed: "Roads are cracked due to the earthquake. Aftershocks are continuing. Evacuation shelters are available at nearby parks."

[0767] The system of the present invention allows users to obtain fast and reliable information in real time and provides feedback that takes into account the user's emotions, thereby realizing a situation in which users can act with peace of mind during events or disasters.

[0768] The processing flow will be explained below.

[0769] Step 1:

[0770] The user opens the dedicated application, writes and attaches the situation at the site in the form of text, images, videos, etc., and presses the send button. Location information and a timestamp are automatically added from the user's device.

[0771] Step 2:

[0772] The server receives the information sent from the user terminal, including the posted text, images, videos, location information, and timestamps.

[0773] Step 3:

[0774] The server uses an algorithm to filter the information it receives, eliminating unnecessary or irrelevant information, and stores the filtered information in a database for analysis.

[0775] Step 4:

[0776] The filtered information is then analyzed by a generation AI on the server, which clusters highly relevant information and groups together information related to the same topic.

[0777] Step 5:

[0778] From the clustered information, the generative AI extracts important keywords, and generates an initial, concise description based on the extracted keywords.

[0779] Step 6:

[0780] The emotion engine analyzes posted text, images, and videos to recognize the user's emotions, such as whether they are excited, confused, or relieved.

[0781] Step 7:

[0782] Based on the user's recognized emotions, the generative AI reflects emotional elements in the description, adding emotional nuance to the description and making it easier to understand.

[0783] Step 8:

[0784] The server examines the generated explanation to identify missing or ambiguous information. The emotion engine takes the user's emotions into account and automatically generates follow-up questions to confirm any additional information needed.

[0785] Step 9:

[0786] The server generates and sends follow-up questions to the user, such as "What's the status of the other stages?" or "Could you please elaborate on the specific situation?"

[0787] Step 10:

[0788] The user terminal receives the additional question, and the user provides an answer (text, image, etc.) to the question. The user presses the send button to send the answer to the server.

[0789] Step 11:

[0790] The server receives additional information from the user and passes it to the generation AI for re-analysis, which regenerates the final description.

[0791] Step 12:

[0792] The server then distributes the resulting description to other users, who then display it in real time as notifications or feeds on their devices, allowing users to obtain fast and accurate information in real time.

[0793] This completes the system that recognizes the user's emotions and provides appropriate information.

[0794] Example 2

[0795] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0796] In modern society, there is a demand for fast and accurate information provision during events and disasters. However, providing information that takes into account the reliability, relevance, and user emotions presents challenges. Specifically, there is a need to quickly extract necessary information from a large amount of mixed information and provide feedback that takes into account the user's emotions.

[0797] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for a user to input and transmit information in the form of text, image, or video via a user terminal, means for automatically adding location information and a timestamp when information is transmitted from the user terminal, means for the server to receive information transmitted from the user terminal, means for filtering the received information using a specific algorithm and eliminating unnecessary information, means for a generation AI to analyze the filtered information and cluster related information, means for the generation AI to extract keywords from the clustered information and generate a concise explanation, means for an emotion engine in the server to recognize the user's emotions, means for the generation AI to reflect emotional elements in the explanation based on the emotion information, means for identifying deficiencies in the generated explanation and automatically generating additional questions to confirm additional information, means for the server to transmit the additional questions to the user terminal, means for a user to provide additional information and the server to receive it, means for regenerating a final explanation based on the additional information received by the server, and means for distributing the final explanation to other users. This will enable users to obtain fast and reliable information in real time during events or disasters, and also provide emotionally sensitive feedback.

[0798] "User terminal" refers to a device that a user uses to input and send information such as text, images, and videos, and includes smartphones, tablets, and computers.

[0799] "Location information" refers to geographic coordinate information obtained using the GPS function of the user's device, and indicates the specific location from which the user sent the information.

[0800] A "timestamp" is information that records the exact date and time when a user sends information, and is an important element when organizing and analyzing information later.

[0801] A "server" refers to a system that receives information sent from user terminals and performs processes such as analysis, filtering, clustering, and regeneration, and operates in cooperation with a large number of devices.

[0802] "Filtering" refers to the process of removing unnecessary or irrelevant information from received information and selecting only highly relevant information.

[0803] An "algorithm" refers to a set of computational or processing steps for solving a specific problem, and is a technique used for filtering and clustering information.

[0804] "Generative AI" refers to an artificial intelligence system that analyzes large amounts of data, extracts important keywords, and generates explanatory text.

[0805] "Clustering" refers to the process of grouping related information, and serves to bring together information that belongs to the same theme or category.

[0806] "Keyword extraction" refers to the process of extracting important words and phrases from clustered information, and is a fundamental step in generating concise descriptions.

[0807] An "emotion engine" is a system that analyzes information such as text, images, and videos sent by users and recognizes and judges their emotions.

[0808] A "concise explanation" is a summary sentence created by the generative AI based on clustered information and keywords, and provides information to users in a clear and easy-to-understand format.

[0809] The "additional question" refers to a question that is automatically generated to identify deficiencies in the generated explanation and to ask the user for additional information.

[0810] The "final description" refers to the complete information provided by the AI ​​after incorporating additional information from the user.

[0811] "Distribution" refers to the process of providing the final generated description to other users as a notification or feed, a fast and effective way to communicate information.

[0812] MODE FOR CARRYING OUT THE INVENTION

[0813] The system of the present invention is composed of a user terminal, a server, a generation AI, and an emotion engine. The operation of each element of the system and a specific embodiment of the entire system will be described below.

[0814] Operation on the user device

[0815] First, the user opens a dedicated application. The user writes and attaches information about the situation at the scene in the form of text, images, videos, etc. For example, if the user is at a music festival, they can post something like, "Band X is currently performing on Stage B, and a large audience is gathering." The user's device automatically adds location information and a timestamp to the entered data and sends it to the server.

[0816] Processing on the server

[0817] The server receives information sent from the user's device. The received content includes posted text, images, videos, location information, and timestamps. For accurate processing, the server filters the information using specific algorithms to eliminate unnecessary information. Next, the server's generation AI analyzes the filtered information and clusters related information. From the clustered information, the generation AI extracts important keywords and generates a concise description.

[0818] Processing with the Emotion Engine

[0819] The emotion engine in the server analyzes the information sent from the user's device and recognizes the user's emotions. Specifically, it determines the user's emotions based on the content of the text, facial expressions in images, and audio in videos. Based on this, the generation AI reflects emotional elements in the explanatory text. This makes it possible to provide information that is easier to understand.

[0820] Generate and submit follow-up questions

[0821] The generated explanation is then scrutinized by the server to identify any missing information or ambiguous parts. Taking into account the user's emotions recognized by the emotion engine, follow-up questions are automatically generated to confirm additional information. For example, if the user is confused, a considerate question such as "Could you please explain the specific situation in more detail?" is generated. The server then sends the generated follow-up question to the user's device. The user then provides additional information in response to the question, such as text or images, and sends it back to the server.

[0822] Final information generation and distribution

[0823] The server receives additional information from the user and stores it in a database. The generation AI regenerates the final description based on the newly added information. For example, a specific description such as "Currently, band X is performing on stage B, causing a crowd. Excitement is building on other stages as well" is generated. The server then distributes the final description to other users. The distributed information is displayed on the user's device as notifications or feeds.

[0824] Examples and prompts

[0825] Examples of music festivals

[0826] 1. User A posts at a music festival, "Band X is performing on Stage B."

[0827] 2. The server receives this post and clusters it with other related posts.

[0828] 3. The AI ​​generates a description such as "Currently, band X is performing on stage B, so it is crowded."

[0829] 4. The server's emotion engine recognizes User A's excitement and reflects it in the description.

[0830] 5. User A is asked a further question and answers, "Band △△ is preparing for Stage C."

[0831] 6. The final explanation will be posted as follows: "Currently, band X is performing on Stage B, causing congestion. Band △△ is preparing on Stage C."

[0832] Prompt Sentence Examples

[0833] "What kind of description should be generated if the user is excited at a music festival?"

[0834] Specific examples of disasters

[0835] 1. User B posts, "The roads are cracked due to the earthquake."

[0836] 2. The server receives this post and parses it along with other related posts.

[0837] 3. The AI ​​generates the following explanation: "The roads are cracked due to the earthquake. Aftershocks are continuing."

[0838] 4. The server's emotion engine recognizes User B's confusion and reflects it in the explanation.

[0839] 5. The server asks User B, "Where is the evacuation shelter?"

[0840] 6. User B answers, "The evacuation site is a nearby park."

[0841] 7. Finally, the following message is displayed: "Roads are cracked due to the earthquake. Aftershocks are continuing. Evacuation shelters are available at nearby parks."

[0842] Prompt Sentence Examples

[0843] "What kind of explanatory text should be generated when a user is confused during a disaster?"

[0844] The system of the present invention allows users to obtain fast and reliable information in real time and receive feedback that takes into account their emotions, thereby realizing a situation in which users can act with peace of mind during events or disasters.

[0845] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0846] Step 1: User opens dedicated application

[0847] How it works: The user launches a dedicated application on their device.

[0848] Input: User operation (application startup)

[0849] Output: The application starts and the input screen is displayed.

[0850] Step 2: User inputs and attaches the situation on site and sends it

[0851] How it works: The user enters text into the text box, attaches images and videos, for example, "Band X is currently performing on Stage B," and then presses the send button.

[0852] Input: Text data, image files, video files

[0853] Output: An information packet with automatically added location and timestamp is generated and sent to the server.

[0854] Step 3: The server receives the information

[0855] Operation: The server receives an information packet sent from the user terminal.

[0856] Input: Information packets (text, images, videos, location information, timestamps)

[0857] Output: The received information is stored in a database.

[0858] Step 4: The server filters the information

[0859] How it works: The server uses specific algorithms to filter out unwanted information from the information it receives, for example filtering out spam messages and irrelevant images.

[0860] Input: Incoming data (text, image, video, location, timestamp)

[0861] Output: Filtered and relevant information is extracted.

[0862] Step 5: Server clustering information

[0863] How it works: A generative AI in the server analyzes the filtered information and groups highly relevant information together, for example, all posts about Stage B into one cluster.

[0864] Input: Filtered information

[0865] Output: Clustered information (information within a cluster is integrated)

[0866] Step 6: Generative AI extracts keywords and generates descriptions

[0867] How it works: The generative AI extracts important keywords from the clustered information and generates a concise description based on them. For example, it might generate a description like, "Currently, band X is performing on Stage B, so it may be crowded."

[0868] Input: Clustered information

[0869] Output: Description

[0870] Step 7: The emotion engine recognizes the user's emotions

[0871] How it works: The emotion engine in the server analyzes the information sent by the user and recognizes the user's emotions, for example, determining excitement or confusion from the wording in the text and facial expressions in the images.

[0872] Input: Information sent (text, images, videos)

[0873] Output: User's emotional information

[0874] Step 8: Reflect emotions in your description

[0875] How it works: Based on the emotional information recognized by the emotion engine, the generation AI reflects emotional elements in the description. For example, it adds the expression "There is a large audience and the atmosphere is lively."

[0876] Input: Emotion information, generated description

[0877] Output: Final description reflecting the sentiment

[0878] Step 9: The server generates and sends a challenge requesting additional information.

[0879] How it works: The server examines the generated explanation, identifies missing information, and automatically generates follow-up questions. For example, it generates a question like, "Could you please tell me more about your specific situation?" and sends it to the user's device.

[0880] Input: Final description

[0881] Output: Additional questions

[0882] Step 10: User provides additional information and submits

[0883] How it works: The user responds to a follow-up question from the server by providing additional information, such as text or images, and submitting it.

[0884] Input: Additional information (text, image)

[0885] Output: Additional information is sent to the server.

[0886] Step 11: Server receives additional information and re-aggregates

[0887] How it works: The server receives additional information sent by the user and re-aggregates it with existing data.

[0888] Input: Additional Information

[0889] Output: Complete information aggregated in a database

[0890] Step 12: The generation AI regenerates the final description

[0891] How it works: The AI ​​then regenerates the final description based on the re-aggregated information. For example, it might generate a specific description like, "Currently, band X is performing on stage B, causing congestion. Band Y is preparing on stage C."

[0892] Input: Re-aggregated information

[0893] Output: Final description

[0894] Step 13: The server delivers the final description

[0895] How it works: The server distributes the final description to other users, who then display it as notifications or feeds on their devices.

[0896] Input: Final description

[0897] Output: Notifications and feeds delivered to user devices

[0898] This allows users to obtain fast and reliable information in real time, and provides emotionally sensitive feedback. This system creates an environment where users can act with confidence during events and disasters.

[0899] (Application example 2)

[0900] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0901] The rapid sharing and efficient management of information within a logistics center is extremely important for improving work efficiency and reducing errors. However, there are often delays in collecting information from the field and providing feedback based on that information, which can result in work delays and incorrect deliveries. Furthermore, there is a need to provide optimal feedback based on the emotions and urgency of field workers, but current systems have difficulty in this regard. To solve these issues, a system is needed that quickly and accurately collects field information and provides feedback that takes into account the emotions of workers.

[0902] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0903] In this invention, the server includes means for receiving information transmitted from a user terminal, means for filtering the received information to eliminate unnecessary information, means for analyzing the filtered information and clustering related information, means for extracting keywords from the clustered information and generating a concise explanation, means for analyzing the emotional state of the user using an emotion engine and reflecting the emotion in the explanation, means for identifying deficiencies in the generated explanation and transmitting additional information to the user, means for receiving the additional information and regenerating a final explanation, and means for distributing the final explanation to other users. This improves the efficiency of information sharing and management within the logistics center and enables prompt and appropriate feedback that takes into account the emotions of field workers.

[0904] A "user terminal" is a device used by workers in a logistics center to input and transmit information.

[0905] A "server" is a device that receives information sent from a user terminal, and performs filtering, analysis, clustering, description generation, sentiment analysis, sending additional information, and regenerating final information.

[0906] "Filtering" refers to the process of filtering out unnecessary information from received information and extracting only the necessary information.

[0907] "Clustering" refers to the process of grouping filtered information based on relevance.

[0908] "Keyword extraction" is the process of selecting important elements from clustered information.

[0909] "Description generation" is the process of creating concise, easy-to-understand text based on the important elements obtained through keyword extraction.

[0910] An "emotion engine" is a system that analyzes information input by a user and recognizes the user's emotional state from that information.

[0911] "Sending additional information" refers to an operation of sending a question or request to the user to fill in any gaps in the generated explanation.

[0912] "Receiving additional information" refers to an operation of receiving supplemental information sent by a user.

[0913] "Regeneration of final information" is the process of updating the description again based on additional information to make it more accurate.

[0914] "Delivery of final information" refers to an operation of notifying other users of the finally generated explanation.

[0915] "Emotion analysis" refers to the process of analyzing a user's emotional state through an emotion engine.

[0916] System Configuration

[0917] This invention is a system consisting of a user terminal, a server, a generative AI, and an emotion engine. The user terminal is used by workers in a logistics center and is responsible for inputting on-site information and sending it to the server. The server is a device that receives, processes, analyzes, recognizes emotions, and finally regenerates and distributes explanatory text.

[0918] Operation on the user device

[0919] On the user device, workers use a dedicated smartphone application to input on-site information. Input can be in a variety of formats, including text, images, and videos, and there is a function to input and send specific information, such as "the package label cannot be found in Section B." Location information and timestamps are also automatically added from the user device. This makes it possible to clearly identify when and where each piece of information occurred.

[0920] Processing on the server

[0921] The server receives information sent from the user terminal. The received information is first processed by a filtering algorithm to remove unnecessary or irrelevant information. Next, highly relevant information is clustered and managed in a consolidated manner.

[0922] Generative AI analyzes the clustered information, extracts important keywords, and generates concise descriptions that summarize the filtered information and convert it into an easy-to-understand format.

[0923] Processing with the Emotion Engine

[0924] The emotion engine in the server analyzes the information sent from the user's device and recognizes the worker's emotional state. For example, emotions such as "confusion," "excitement," and "relief" are determined through text and image analysis. This adds emotional elements to the generated explanation, enabling the system to provide more considerate feedback to the user.

[0925] Generate and submit follow-up questions

[0926] The server then carefully examines the initially generated description to identify any missing or ambiguous information. In such cases, the AI ​​takes into account the user's emotions as recognized by the emotion engine and automatically generates questions to request additional information. For example, for a confused user, it generates a thoughtful question such as, "Could you please explain the specific situation in more detail?"

[0927] Final information generation and distribution

[0928] The server receives additional information from the user and stores it in a database. The generation AI regenerates the final explanation based on the newly added information. For example, a specific explanation such as "The package label cannot be confirmed in section B. Upon checking, we found that the label was missing, so we attached a new label" is generated. The final explanation is notified to other workers and managers, enabling information sharing in real time.

[0929] Specific examples

[0930] For example:

[0931] Example 1: Information sharing in a logistics center

[0932] 1. Worker A types "I can't find the package label in Section B" into his smartphone and sends it.

[0933] 2. The server receives this information and performs filtering and clustering.

[0934] 3. The AI ​​generates the explanation "The package label cannot be confirmed in section B."

[0935] 4. The emotion engine recognizes worker A's confusion and reflects it in the explanation.

[0936] 5. A follow-up question is generated and sent to worker A: "Could you please tell me more about the specific situation?"

[0937] 6. Worker A responds, "I checked and found that the label was missing, so I attached a new label."

[0938] 7. The final explanation is sent to other workers: "Section B could not find the package label. Upon checking, we found that the label was missing, so we attached a new label."

[0939] Prompt Sentence Examples

[0940] "User Input: Package label not found in Section B."

[0941] "Generative AI model prompt: Please describe the situation on the ground."

[0942] "Sentiment analysis: puzzled"

[0943] "Final description: Problem with section B. Details: Unable to locate package label. Emotion: Confused."

[0944] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0945] Step 1:

[0946] Entering information on the user's device

[0947] A worker with a user device reports a specific problem or situation at the logistics center. The worker opens a dedicated smartphone application and inputs the situation in the form of text, images, or videos, such as "The package label cannot be found in Section B." The user device automatically adds location information and a timestamp.

[0948] Input: On-site information (text, images, videos), location information, timestamp

[0949] Output: Structured information data

[0950] Step 2:

[0951] Receiving and filtering information

[0952] The server receives the information sent from the user's device and filters it using specific algorithms to remove unnecessary or irrelevant information, such as typos and meaningless data from the text.

[0953] Input: Structured information data

[0954] Output: Filtered information data

[0955] Step 3:

[0956] Information Clustering

[0957] The server analyzes the filtered information and clusters related information, for example, all "questions related to Section B" into one cluster, thereby aggregating information for specific sections or situations.

[0958] Input: Filtered information data

[0959] Output: Clustered information data

[0960] Step 4:

[0961] Keyword extraction and description generation

[0962] The server's generation AI extracts important keywords from the clustered information and generates a concise explanation, such as "The luggage label cannot be found in section B."

[0963] Input: Clustered information data

[0964] Output: Generated description

[0965] Step 5:

[0966] Emotion recognition

[0967] The server uses an emotion engine to analyze information sent from the user's device and recognize the worker's emotional state. For example, it can determine whether the worker is in a "confused" state from text and images.

[0968] Input: Generated description, information from user terminal

[0969] Output: Recognized emotion data

[0970] Step 6:

[0971] Generate and submit follow-up questions

[0972] If the server determines that the generated description is insufficient, the AI ​​generates additional questions based on the emotion recognition results, such as "Could you please tell me more about the specific situation?"

[0973] Input: Generated description, recognized emotion data

[0974] Output: Additional questions

[0975] Step 7:

[0976] Receiving additional information from the user

[0977] The user terminal receives the query from the server, and the worker enters specific additional information and resubmits it. For example, the worker returns information such as, "After checking, I found that a label was missing, so I attached a new label."

[0978] Input: Additional Question

[0979] Output: Additional information data

[0980] Step 8:

[0981] Regenerate the final information

[0982] The server re-analyzes the generated AI based on the additional information and re-generates the final description, for example, "The package label cannot be found in section B. Upon checking, the label was found to be missing, so a new label was attached."

[0983] Input: Additional information data

[0984] Output: Final description

[0985] Step 9:

[0986] Final information distribution

[0987] The server notifies other users of the final generated explanation, for example, by distributing it in real time to other workers or administrators.

[0988] Input: Final description

[0989] Output: Notification description

[0990] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0991] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0992] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0993] [Third embodiment]

[0994] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0995] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0996] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0997] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0998] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0999] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1000] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1001] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1002] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1003] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1004] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1005] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1006] The system of the present invention is composed of a user terminal, a server, and a generation AI. Below, we will provide a detailed explanation of how each element operates and how the entire system functions.

[1007] Operation on the user device

[1008] Enter and submit information

[1009] When a user is at an event or disaster site, they open a dedicated application. They write and attach information about the situation at the site in the form of text, images, or videos, and then press the send button. For example, they could post something like, "Band X is currently performing on Stage B, and a large audience is gathering." At this time, location information and a timestamp are automatically added from the user's device.

[1010] Processing on the server

[1011] Receiving and filtering information

[1012] The server receives information sent from the user's device. The received information includes the content of the post, location information, timestamp, etc. After receiving the information, the server filters it using a specific algorithm to eliminate unnecessary or irrelevant information. The relevant information is then passed on to the next processing step.

[1013] Information clustering and keyword extraction

[1014] The filtered information is analyzed by a generation AI in the server, which clusters the filtered information into related groups (e.g., all information related to Stage B into one cluster).The generation AI then extracts important keywords from each cluster and generates a concise description.

[1015] Examining the generated explanation and generating follow-up questions

[1016] The server then carefully examines the generated description to identify any missing information or ambiguities. For example, if the generated description is "Stage B is currently crowded because band X is playing," the server will automatically generate additional questions about the situation on other stages and the number of audience members.

[1017] Asking the user additional questions and completing information

[1018] Submit and answer additional questions

[1019] The server generates a follow-up question and sends it to the user. For example, a question like "What is the status of the other stages?" is sent to the user. The user responds to the question by providing additional information in the form of text or images. This answer is then sent back to the server.

[1020] Final information generation and distribution

[1021] Re-aggregating information and regenerating descriptions

[1022] The server receives additional information from the user and stores it in a database. The generation AI then regenerates the final description based on the newly added information. For example, a specific description such as "Currently, band X is performing on stage B, causing congestion. Band Y is preparing on stage C." is generated.

[1023] Final information distribution

[1024] The server then distributes the resulting descriptions to other users in real time, who then display the information as notifications or feeds.

[1025] Specific examples

[1026] Music festival scenario

[1027] 1. User A posts at a music festival, "Band X is performing on Stage B."

[1028] 2. The server receives this post and clusters it with other related posts.

[1029] 3. The AI ​​generates a description like "Stage B is currently crowded because band X is performing," but it lacks information about the other stages.

[1030] 4. The server sends a follow-up question to User A: "What's the status of the other stages?"

[1031] 5. User A answers, "Band △△ is preparing for stage C."

[1032] 6. The final description will be "Currently, stage B is crowded as band X is performing. band YY is preparing on stage C."

[1033] Disaster scenarios

[1034] 1. User B posts, "The roads are cracked due to the earthquake."

[1035] 2. The server receives this post and parses it along with other related posts.

[1036] 3. The AI ​​generates the description "Roads are cracked due to the earthquake. Aftershocks are continuing," but it lacks information about evacuation sites.

[1037] 4. The server asks User B an additional question: "Where is the evacuation site?"

[1038] 5. User B answers, "The evacuation site is a nearby park."

[1039] 6. Finally, the following message is displayed: "Roads are cracked due to the earthquake. Aftershocks are continuing. Evacuation shelter is in a nearby park."

[1040] The system of the present invention allows users to obtain fast and reliable information in real time, enabling accurate situational understanding even during events or disasters.

[1041] The processing flow will be explained below.

[1042] Step 1:

[1043] The user opens the dedicated application, writes and attaches the situation at the site in the form of text, images, videos, etc., and presses the send button. Location information and a timestamp are automatically added from the user's device.

[1044] Step 2:

[1045] The server receives the information sent from the user terminal, including the posted text, images, videos, location information, and timestamps.

[1046] Step 3:

[1047] The server filters the received information using specific algorithms to remove unnecessary or irrelevant information, and stores the filtered information for analysis.

[1048] Step 4:

[1049] Based on the filtered information, the generative AI clusters this information, specifically grouping highly related information together.

[1050] Step 5:

[1051] From the clustered information, the generative AI extracts important keywords, and generates an initial, concise description based on the extracted keywords.

[1052] Step 6:

[1053] The server reviews the generated description to identify any missing or ambiguous information, confirms the additional information needed, and proceeds to the next step.

[1054] Step 7:

[1055] The server automatically generates additional questions based on the missing information and sends the generated questions to the corresponding user terminal.

[1056] Step 8:

[1057] The user terminal receives the additional question, and the user provides an answer (text, image, etc.) to the question. The user presses the send button to send the answer to the server.

[1058] Step 9:

[1059] The server receives additional information from the user and passes it back to the AI ​​for re-analysis, which then generates the final description.

[1060] Step 10:

[1061] The server then distributes the final description to other users, who then display it in real time as notifications or feeds on their devices.

[1062] This allows users to obtain fast and accurate information in real time.

[1063] Example 1

[1064] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1065] During modern events and disasters, it is important to quickly and accurately grasp the situation at the scene, but manually collecting and organizing information takes time and effort. It is also difficult to ensure the reliability of the information and real-time availability of the information. Furthermore, when efficiently processing multi-format data (text, images, and video) and converting it into easy-to-understand descriptions, additional information from users is often not reflected in a timely manner. To address these issues, effective information reception, filtering, clustering, keyword extraction, and improved accuracy of automatically generated descriptions are required.

[1066] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1067] In this invention, the server includes means for receiving multi-format data transmitted from a user terminal, means for initially processing the received data and eliminating irrelevant data, means for clustering the filtered data and organizing it into related information groups, means for extracting keywords from the clustered information groups and generating an explanatory text, means for identifying missing information in the generated explanatory text and requesting additional information from the user, means for receiving the additional information from the user and regenerating a final explanatory text, and means for distributing the final explanatory text to other users in real time, thereby enabling efficient processing of multi-format data and providing reliable information in real time.

[1068] A "user terminal" is a computing device that a user uses to input and transmit information, and includes smartphones, tablets, laptops, and the like.

[1069] "Multi-format data" refers to digital data in different formats, such as text, images, and video.

[1070] A "server" is a computer system that receives, processes, stores, and distributes data sent from user terminals over a network.

[1071] "Initial processing" refers to processing to analyze received data and eliminate signal noise and unnecessary information.

[1072] "Filtering" refers to the process of removing unnecessary information from received data and extracting highly relevant information.

[1073] "Clustering" is a technical method for grouping data that have similar characteristics.

[1074] An "information group" is a collection of a series of clustered related information.

[1075] "Keyword extraction" is the process of identifying and extracting important words and phrases from text data.

[1076] "Explanation" refers to text information generated in a format that is easy for users to understand.

[1077] A "generative AI model" is an algorithm or model that uses artificial intelligence to analyze data, extract keywords, and generate descriptions.

[1078] "Additional information" refers to information requested from the user to reacquire data that was missing at the time of initial information collection.

[1079] "Real-time delivery" is the process of providing the results to other users immediately at the moment the data is collected and analyzed.

[1080] The system of the present invention is composed of a user terminal, a server, and a generative AI model. Below, we will explain in detail how each element operates and how the entire system functions.

[1081] Hardware and Software Configuration

[1082] User device:

[1083] A user device is a computing device used by a user, and includes smartphones, tablets, laptops, etc. Using these devices, users open a dedicated application and enter information about an event or disaster site in the form of text, images, and videos. For example, a user might enter text such as "Band X is currently performing on Stage B," attach a photo, and press the send button. The user device automatically adds location information and a timestamp before sending the information.

[1084] server:

[1085] The server is a computer system that receives and processes multi-format data sent from user devices via a network. The received data is stored in a real-time database, and irrelevant and inconsistent data is eliminated as an initial process. The server then filters the data using a specific algorithm to extract only highly relevant data. The filtered data is then clustered using a generative AI model and organized into related information groups. Machine learning techniques are used for this clustering process. The generative AI model then extracts keywords from each cluster and generates a concise explanation to be communicated to the user.

[1086] Generative AI models:

[1087] The generative AI model is an algorithm that extracts keywords from the clustered information and generates explanatory text. If the generated explanation is incomplete, the server identifies this and generates and sends a prompt to the user requesting additional information.

[1088] Specific examples

[1089] Music festival scenario

[1090] 1. User A is at a music festival and types in the text "XX band is performing on Stage B," attaches a photo of the stage, and presses the send button.

[1091] 2. The server receives this information and stores the data, including location and timestamp, in a real-time database.

[1092] 3. A filtering algorithm is applied to remove unnecessary data.

[1093] 4. The generative AI model clusters related information and combines information related to "Stage B" into a single cluster.

[1094] 5. The generative AI model extracts keywords from the cluster and generates a description such as, "Stage B is crowded because band X is performing."

[1095] 6. The server generates a follow-up question: "What is the status of the other stages?" and notifies User A.

[1096] 7. User A replies, "Band △△ is being prepared on Stage C," and the information is sent again.

[1097] 8. The server receives the additional information, and the generative AI model regenerates the final description: "Currently, stage B is crowded because band X is performing. On stage C, band YY is preparing."

[1098] 9. The server distributes the final information to other users in real time.

[1099] Disaster scenarios

[1100] 1. User B posts, "The roads are cracked due to the earthquake," attaches a photo of the scene, and presses the send button.

[1101] 2. The server receives this information and stores it in a real-time database.

[1102] 3. A filtering algorithm is applied to extract only the required data.

[1103] 4. The generative AI model clusters related information and combines information about "earthquakes" into one cluster.

[1104] 5. The generative AI model extracts keywords from the cluster and generates a description such as, "The roads are cracked due to the earthquake. Aftershocks are continuing."

[1105] 6. The server generates an additional question, "Where is the evacuation site?" and notifies User B.

[1106] 7. User B replies, "The evacuation site is a nearby park," and the information is sent again.

[1107] 8. The server receives the additional information, and the generative AI model regenerates the final description: "Roads are cracked due to the earthquake. Aftershocks are continuing. Evacuation shelters are in nearby parks."

[1108] 9. The server distributes the final information to other users in real time.

[1109] This allows users to obtain fast and reliable information in real time, enabling accurate situational understanding even during events or disasters.

[1110] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1111] Step 1:

[1112] The user opens a dedicated application. The user enters information about the event or disaster site in text, image, and video format, and presses the "Send" button. At this time, the user's device automatically adds location information and a timestamp. A specific example of how this works is that the user enters "Band X is currently performing on Stage B," attaches a photo of the scene, and presses the "Send" button. The input data consists of text, images, location information, and a timestamp, and is then sent to the server.

[1113] Step 2:

[1114] The server receives multi-format data sent from user devices. The server performs initial processing of the received data and applies filtering algorithms to remove irrelevant or inconsistent data. The input data is all the information sent by the user, and the output data is clean data with unnecessary data removed. Specific operations include removing spam messages and checking format consistency.

[1115] Step 3:

[1116] The server clusters the filtered data. Using a generative AI model, related information is grouped and organized into related information clusters. The input data is the filtered data, and the output data is the clustered information clusters. Specifically, multiple posts about the same event are grouped into one cluster.

[1117] Step 4:

[1118] The server uses a generative AI model to extract keywords from the clustered information and generate a description. The input data is the clustered information, and the output data is the generated description. For example, a description such as "Stage B is crowded because band X is performing" is automatically generated.

[1119] Step 5:

[1120] The server examines the generated description and identifies any missing or ambiguous information. Based on this, the server generates a prompt to request additional information from the user. The input data is the generated description, and the output data is a prompt for additional information. Specifically, a question such as "What is the status of the other stages?" is generated.

[1121] Step 6:

[1122] The server notifies the user terminal of the generated prompt text. The user receives the notification and inputs additional information. This additional information is also sent in the form of text, images, or video. The input data is the additional information sent by the user, and the output data is additional data including the additional information. A specific operation is for the user to reply, "△△ band is being prepared on stage C," and send it.

[1123] Step 7:

[1124] The server receives additional information from the user and regenerates the final description using the generative AI model. The input data is the additional information from the user, and the output data is the regenerated final description. A specific example of how this works is that the final description might read, "Currently, band X is performing on stage B, causing congestion. Band Y is preparing on stage C."

[1125] Step 8:

[1126] The server distributes the generated final description to other users in real time. The user's device displays the final information in the form of a notification or feed. The input data is the final description, and the output data is the distributed notification. Specifically, the smartphone notification screen will display "Currently, band X is performing on stage B, causing congestion. Band Y is preparing on stage C."

[1127] This allows the system to efficiently process information from users and provide other users with reliable information in real time.

[1128] (Application example 1)

[1129] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1130] In modern society, it is extremely important to quickly and accurately collect information about abnormal situations and emergencies and notify relevant parties. However, current information collection systems often contain a lot of unnecessary information and lack real-time capabilities, making it difficult to respond quickly. Information is also often insufficient or ambiguous, making it difficult to collect additional information. This can hinder appropriate responses in emergencies. It is necessary to solve these problems and achieve efficient and rapid reporting and response to abnormal situations.

[1131] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1132] In this invention, the server includes: means for receiving information transmitted from a user terminal; means for filtering the received information to eliminate unnecessary information; means for analyzing the filtered information and clustering relevant information; means for extracting keywords from the clustered information and generating a concise explanation; means for identifying deficiencies in the generated explanation and transmitting additional information to the user; means for receiving the additional information and regenerating a final explanation; means for distributing the final explanation to other users; means for comprehensively analyzing the anomaly report information transmitted from the user and detecting special circumstances; and means for automatically generating follow-up questions based on the special circumstances and transmitting them to the user in real time. This enables the server to quickly and accurately collect and analyze anomaly information reported by users and efficiently supplement missing information. The generated information can then be instantly distributed to relevant parties, enabling a rapid response.

[1133] A "user terminal" is an electronic device operated by a user, and is a terminal device for inputting and transmitting information.

[1134] "Information" refers to all data sent from a user terminal, such as text, images, videos, location information, and timestamps.

[1135] "Means for receiving" refers to a mechanism for receiving and storing information sent from a user terminal.

[1136] A "filtering means" is a mechanism for filtering out unnecessary or irrelevant information from received information.

[1137] The "means for analyzing and clustering related information" is a method for analyzing the filtered information and grouping related information into one group.

[1138] "Means for extracting keywords and generating concise descriptions" is a technology for extracting important words and phrases from clustered information and creating concise sentences based on them.

[1139] The "means for identifying missing parts of the generated description and sending additional information to the user" is a mechanism for verifying the generated description, identifying missing information, and inquiring about that information from the user.

[1140] The "means for receiving additional information and regenerating the final description" is a method for creating a new, compiled final description based on the additional information provided by the user.

[1141] The "means for distributing the final description to other users" is a mechanism for transmitting the generated final description to other users in real time.

[1142] "Anomaly report information" refers to information reported by a user regarding an abnormality or emergency.

[1143] The "means for detecting special situations" is a method for analyzing abnormality report information and identifying abnormal situations based on specific conditions or patterns.

[1144] The "means for automatically generating follow-up questions and sending them to the user in real time" is a mechanism for generating necessary follow-up questions based on the detected abnormal situation and sending them to the user immediately.

[1145] The system of the present invention is composed of a user terminal, a server, and a generation AI. Below, we will provide a detailed explanation of how each element operates and how the entire system functions.

[1146] Operation on the user device

[1147] Enter and submit information:

[1148] Users use a dedicated application installed on their smartphones or smart glasses. When they witness an abnormal or emergency situation, they record the situation on the scene in the form of text, images, or videos, and press the send button. At this time, location information and a timestamp are automatically added. For example, a user might post, "I spotted a suspicious person. I'm attaching a photo."

[1149] Processing on the server

[1150] Receiving and filtering information:

[1151] The server receives the information sent from the user's device and uses the server's computing resources (e.g., AWS, Google Cloud Platform) and a filtering algorithm to remove unnecessary data from the received information. For example, it performs processing such as "excluding information about suspicious individuals."

[1152] Information clustering and keyword extraction:

[1153] The filtered information is analyzed by a generative AI (e.g., a GPT model) on the server. The generative AI uses a clustering algorithm to group related information together. It then extracts important keywords and generates a concise description. For example, it creates a description such as, "A suspicious person has been spotted in a specific location. We would like to report the current situation."

[1154] Examining the generated explanation and generating follow-up questions:

[1155] The server then examines the generated description to identify any missing information or ambiguity. For example, if the generated description is "A suspicious person was spotted in a specific location," the server automatically generates follow-up questions such as "What is the situation in other areas?". The server uses a generative AI model and prompts to generate follow-up questions.

[1156] Asking the user additional questions and completing information

[1157] Submit and answer additional questions:

[1158] The server generates a follow-up question and sends it to the user. For example, the user might ask, "What is the situation in other areas?" The user then provides additional information in the form of text or images. This answer is then sent back to the server.

[1159] Final information generation and distribution

[1160] Re-aggregate information and regenerate descriptions:

[1161] The server receives additional information from the user and stores it in a database. The AI ​​then regenerates the final description based on the newly added information. For example, a specific description such as "A suspicious person was spotted in a specific location. Other areas are being investigated."

[1162] Final information distribution:

[1163] The server then distributes the resulting description to other users in real time, and the user's device displays this information as a notification or feed, enabling fast and accurate information sharing.

[1164] Examples of concrete examples and prompts

[1165] Examples:

[1166] 1. A user reports that they have spotted a suspicious person in a specific location.

[1167] 2. The server receives the reports and filters and clusters the relevant information.

[1168] 3. The AI ​​generates a description that says "A suspicious person was spotted in a specific location," but lacks information about other areas.

[1169] 4. The server sends a follow-up question to the user: "What's the situation in other areas?"

[1170] 5. The user replies, "There are no abnormalities in other areas."

[1171] 6. The final description will be "Suspicious person spotted in specific location. No abnormalities in other areas."

[1172] Example prompt sentence:

[1173] A user reports a suspicious person in a specific location. Please suggest the next course of action based on the current situation on the scene.

[1174] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1175] Step 1:

[1176] Entering and sending information on the user's device

[1177] When a user witnesses an abnormal or emergency situation, they input information using a dedicated application installed on their smartphone or smart glasses. The input information is recorded in text, image, or video format and sent to a server by pressing the send button. Location information and a timestamp are automatically added.

[1178] Input: Text, images, videos, with automatically added location and timestamps

[1179] Output: Anomaly information sent to the server (text, images, videos, location information, timestamp)

[1180] Step 2:

[1181] Receiving and filtering information on the server

[1182] The server receives the anomaly information sent from the user terminal, and then uses a filtering algorithm to filter out unnecessary or irrelevant information.

[1183] Input: Anomaly information submitted by the user (text, image, video, location, timestamp)

[1184] Output: Filtered and relevant anomaly information

[1185] Step 3:

[1186] Server-based information clustering and keyword extraction

[1187] The filtered information is analyzed using generative AI (e.g., GPT model) to cluster relevant information, and then important keywords are extracted from the clustered information to generate a concise description.

[1188] Input: Filtered anomaly information

[1189] Output: Generated description (e.g. "A suspicious person was spotted at a specific location")

[1190] Step 4:

[1191] The server examines the generated explanation and generates follow-up questions

[1192] The server then examines the generated explanation to identify any missing information or ambiguity. It then automatically generates follow-up questions using a generative AI model and creates prompts to notify the user.

[1193] Input: Generated description

[1194] Output: Prompt for follow-up questions (e.g., "What about other areas?")

[1195] Step 5:

[1196] Sending and answering follow-up questions to users

[1197] The server generates a follow-up question and sends it to the user, who then provides additional information in the form of text, images, etc. The answer is then sent back to the server.

[1198] Input: Prompt for follow-up questions, additional information from the user (text, images, etc.)

[1199] Output: Additional information sent to the server

[1200] Step 6:

[1201] Server re-aggregates information and regenerates descriptions

[1202] The server receives additional information from the user and stores it in a database. The AI ​​then regenerates the final description based on the newly added information.

[1203] Input: Additional information from the user

[1204] Output: Final description (e.g. "Suspicious individuals were spotted in specific locations. No anomalies found in other areas.")

[1205] Step 7:

[1206] Final information generation and distribution

[1207] The server then distributes the resulting description to other users in real time, and the user's device displays this information as notifications or feeds.

[1208] Input: Final description

[1209] Output: Notifications or feeds delivered to other users

[1210] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1211] The system of the present invention is composed of a user terminal, a server, a generation AI, and an emotion engine. Below, we will provide a detailed explanation of how each element operates and how the entire system functions.

[1212] Operation on the user device

[1213] Enter and submit information

[1214] When a user is at an event or disaster site, they open a dedicated application. They write and attach information about the situation at the site in the form of text, images, or videos, and then press the send button. For example, they can post content such as "Band X is currently performing on Stage B, and a large audience is gathering." Location information and a timestamp are also automatically added from the user's device.

[1215] Processing on the server

[1216] Receiving and filtering information

[1217] The server receives the information sent from the user's device. The received content includes the posted text, images, videos, location information, and timestamps. After receiving the information, the server filters it using a specific algorithm to eliminate unnecessary or irrelevant information. The relevant information is then passed on to the next processing step.

[1218] Information clustering and keyword extraction

[1219] The filtered information is analyzed by a generation AI in the server, which clusters the filtered information into related groups (e.g., all information related to Stage B into one cluster).The generation AI then extracts important keywords from each cluster and generates a concise description.

[1220] Processing with the Emotion Engine

[1221] Emotion recognition

[1222] The emotion engine in the server analyzes information (text, images, videos, etc.) sent from the user's device and recognizes the user's emotions. For example, it determines whether the user is excited, confused, or relieved based on the wording in the text and facial expressions in the images.

[1223] Reflecting emotion in descriptions

[1224] Based on the user's emotional information recognized by the emotion engine, the generation AI reflects emotional elements in the explanation, adding emotional nuance to the explanation, making it possible to provide information that is easier to understand.

[1225] Generate and submit follow-up questions

[1226] Generate follow-up questions

[1227] The server then carefully examines the initially generated description to identify any missing information or ambiguities. Taking into account the user's emotions as recognized by the emotion engine, follow-up questions are automatically generated to confirm any additional information needed. For example, if the user is confused, a considerate question such as "Could you please explain the specific situation in more detail?" is generated.

[1228] Submit and answer questions

[1229] The server generates a follow-up question and sends it to the user, who then provides additional information in the form of text, images, etc. The answer is then sent back to the server.

[1230] Final information generation and distribution

[1231] Re-aggregating information and regenerating descriptions

[1232] The server receives additional information from the user and stores it in a database. The AI ​​then regenerates the final description based on the newly added information. For example, a specific description such as "Currently, band X is performing on stage B, and it's crowded. Excitement is building on the other stages as well."

[1233] Final information distribution

[1234] The server then distributes the resulting description to other users, who then display the information as notifications or feeds.

[1235] Specific examples

[1236] In the case of a music festival

[1237] 1. User A posts at a music festival, "Band X is performing on Stage B."

[1238] 2. The server receives this post and clusters it with other related posts.

[1239] 3. The AI ​​generates a description such as "Currently, band X is performing on stage B, so it is crowded."

[1240] 4. The server's emotion engine recognizes User A's excitement and reflects it in the description.

[1241] 5. User A is asked a further question and answers, "Band △△ is preparing for Stage C."

[1242] 6. The final explanation will be posted as follows: "Currently, band X is performing on Stage B, causing congestion. Band △△ is preparing on Stage C."

[1243] In case of disaster

[1244] 1. User B posts, "The roads are cracked due to the earthquake."

[1245] 2. The server receives this post and parses it along with other related posts.

[1246] 3. The AI ​​generates the following explanation: "The roads are cracked due to the earthquake. Aftershocks are continuing."

[1247] 4. The server's emotion engine recognizes User B's confusion and reflects it in the explanation.

[1248] 5. The server asks User B, "Where is the evacuation shelter?"

[1249] 6. User B answers, "The evacuation site is a nearby park."

[1250] 7. Finally, the following message is displayed: "Roads are cracked due to the earthquake. Aftershocks are continuing. Evacuation shelters are available at nearby parks."

[1251] The system of the present invention allows users to obtain fast and reliable information in real time and provides feedback that takes into account the user's emotions, thereby realizing a situation in which users can act with peace of mind during events or disasters.

[1252] The processing flow will be explained below.

[1253] Step 1:

[1254] The user opens the dedicated application, writes and attaches the situation at the site in the form of text, images, videos, etc., and presses the send button. Location information and a timestamp are automatically added from the user's device.

[1255] Step 2:

[1256] The server receives the information sent from the user terminal, including the posted text, images, videos, location information, and timestamps.

[1257] Step 3:

[1258] The server uses an algorithm to filter the information it receives, eliminating unnecessary or irrelevant information, and stores the filtered information in a database for analysis.

[1259] Step 4:

[1260] The filtered information is then analyzed by a generation AI on the server, which clusters highly relevant information and groups together information related to the same topic.

[1261] Step 5:

[1262] From the clustered information, the generative AI extracts important keywords, and generates an initial, concise description based on the extracted keywords.

[1263] Step 6:

[1264] The emotion engine analyzes posted text, images, and videos to recognize the user's emotions, such as whether they are excited, confused, or relieved.

[1265] Step 7:

[1266] Based on the user's recognized emotions, the generative AI reflects emotional elements in the description, adding emotional nuance to the description and making it easier to understand.

[1267] Step 8:

[1268] The server examines the generated explanation to identify missing or ambiguous information. The emotion engine takes the user's emotions into account and automatically generates follow-up questions to confirm any additional information needed.

[1269] Step 9:

[1270] The server generates and sends follow-up questions to the user, such as "What's the status of the other stages?" or "Could you please elaborate on the specific situation?"

[1271] Step 10:

[1272] The user terminal receives the additional question, and the user provides an answer (text, image, etc.) to the question. The user presses the send button to send the answer to the server.

[1273] Step 11:

[1274] The server receives additional information from the user and passes it to the generation AI for re-analysis, which regenerates the final description.

[1275] Step 12:

[1276] The server then distributes the resulting description to other users, who then display it in real time as notifications or feeds on their devices, allowing users to obtain fast and accurate information in real time.

[1277] This completes the system that recognizes the user's emotions and provides appropriate information.

[1278] Example 2

[1279] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1280] In modern society, there is a demand for fast and accurate information provision during events and disasters. However, providing information that takes into account the reliability, relevance, and user emotions presents challenges. Specifically, there is a need to quickly extract necessary information from a large amount of mixed information and provide feedback that takes into account the user's emotions.

[1281] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for a user to input and transmit information in the form of text, image, or video via a user terminal, means for automatically adding location information and a timestamp when information is transmitted from the user terminal, means for the server to receive information transmitted from the user terminal, means for filtering the received information using a specific algorithm and eliminating unnecessary information, means for a generation AI to analyze the filtered information and cluster related information, means for the generation AI to extract keywords from the clustered information and generate a concise explanation, means for an emotion engine in the server to recognize the user's emotions, means for the generation AI to reflect emotional elements in the explanation based on the emotion information, means for identifying deficiencies in the generated explanation and automatically generating additional questions to confirm additional information, means for the server to transmit the additional questions to the user terminal, means for a user to provide additional information and the server to receive it, means for regenerating a final explanation based on the additional information received by the server, and means for distributing the final explanation to other users. This will enable users to obtain fast and reliable information in real time during events or disasters, and also provide emotionally sensitive feedback.

[1282] "User terminal" refers to a device that a user uses to input and send information such as text, images, and videos, and includes smartphones, tablets, and computers.

[1283] "Location information" refers to geographic coordinate information obtained using the GPS function of the user's device, and indicates the specific location from which the user sent the information.

[1284] A "timestamp" is information that records the exact date and time when a user sends information, and is an important element when organizing and analyzing information later.

[1285] A "server" refers to a system that receives information sent from user terminals and performs processes such as analysis, filtering, clustering, and regeneration, and operates in cooperation with a large number of devices.

[1286] "Filtering" refers to the process of removing unnecessary or irrelevant information from received information and selecting only highly relevant information.

[1287] An "algorithm" refers to a set of computational or processing steps for solving a specific problem, and is a technique used for filtering and clustering information.

[1288] "Generative AI" refers to an artificial intelligence system that analyzes large amounts of data, extracts important keywords, and generates explanatory text.

[1289] "Clustering" refers to the process of grouping related information, and serves to bring together information that belongs to the same theme or category.

[1290] "Keyword extraction" refers to the process of extracting important words and phrases from clustered information, and is a fundamental step in generating concise descriptions.

[1291] An "emotion engine" is a system that analyzes information such as text, images, and videos sent by users and recognizes and judges their emotions.

[1292] A "concise explanation" is a summary sentence created by the generative AI based on clustered information and keywords, and provides information to users in a clear and easy-to-understand format.

[1293] The "additional question" refers to a question that is automatically generated to identify deficiencies in the generated explanation and to ask the user for additional information.

[1294] The "final description" refers to the complete information provided by the AI ​​after incorporating additional information from the user.

[1295] "Distribution" refers to the process of providing the final generated description to other users as a notification or feed, a fast and effective way to communicate information.

[1296] MODE FOR CARRYING OUT THE INVENTION

[1297] The system of the present invention is composed of a user terminal, a server, a generation AI, and an emotion engine. The operation of each element of the system and a specific embodiment of the entire system will be described below.

[1298] Operation on the user device

[1299] First, the user opens a dedicated application. The user writes and attaches information about the situation at the scene in the form of text, images, videos, etc. For example, if the user is at a music festival, they can post something like, "Band X is currently performing on Stage B, and a large audience is gathering." The user's device automatically adds location information and a timestamp to the entered data and sends it to the server.

[1300] Processing on the server

[1301] The server receives information sent from the user's device. The received content includes posted text, images, videos, location information, and timestamps. For accurate processing, the server filters the information using specific algorithms to eliminate unnecessary information. Next, the server's generation AI analyzes the filtered information and clusters related information. From the clustered information, the generation AI extracts important keywords and generates a concise description.

[1302] Processing with the Emotion Engine

[1303] The emotion engine in the server analyzes the information sent from the user's device and recognizes the user's emotions. Specifically, it determines the user's emotions based on the content of the text, facial expressions in images, and audio in videos. Based on this, the generation AI reflects emotional elements in the explanatory text. This makes it possible to provide information that is easier to understand.

[1304] Generate and submit follow-up questions

[1305] The generated explanation is then scrutinized by the server to identify any missing information or ambiguous parts. Taking into account the user's emotions recognized by the emotion engine, follow-up questions are automatically generated to confirm additional information. For example, if the user is confused, a considerate question such as "Could you please explain the specific situation in more detail?" is generated. The server then sends the generated follow-up question to the user's device. The user then provides additional information in response to the question, such as text or images, and sends it back to the server.

[1306] Final information generation and distribution

[1307] The server receives additional information from the user and stores it in a database. The generation AI regenerates the final description based on the newly added information. For example, a specific description such as "Currently, band X is performing on stage B, causing a crowd. Excitement is building on other stages as well" is generated. The server then distributes the final description to other users. The distributed information is displayed on the user's device as notifications or feeds.

[1308] Examples and prompts

[1309] Examples of music festivals

[1310] 1. User A posts at a music festival, "Band X is performing on Stage B."

[1311] 2. The server receives this post and clusters it with other related posts.

[1312] 3. The AI ​​generates a description such as "Currently, band X is performing on stage B, so it is crowded."

[1313] 4. The server's emotion engine recognizes User A's excitement and reflects it in the description.

[1314] 5. User A is asked a further question and answers, "Band △△ is preparing for Stage C."

[1315] 6. The final explanation will be posted as follows: "Currently, band X is performing on Stage B, causing congestion. Band △△ is preparing on Stage C."

[1316] Prompt Sentence Examples

[1317] "What kind of description should be generated if the user is excited at a music festival?"

[1318] Specific examples of disasters

[1319] 1. User B posts, "The roads are cracked due to the earthquake."

[1320] 2. The server receives this post and parses it along with other related posts.

[1321] 3. The AI ​​generates the following explanation: "The roads are cracked due to the earthquake. Aftershocks are continuing."

[1322] 4. The server's emotion engine recognizes User B's confusion and reflects it in the explanation.

[1323] 5. The server asks User B, "Where is the evacuation shelter?"

[1324] 6. User B answers, "The evacuation site is a nearby park."

[1325] 7. Finally, the following message is displayed: "Roads are cracked due to the earthquake. Aftershocks are continuing. Evacuation shelters are available at nearby parks."

[1326] Prompt Sentence Examples

[1327] "What kind of explanatory text should be generated when a user is confused during a disaster?"

[1328] The system of the present invention allows users to obtain fast and reliable information in real time and receive feedback that takes into account their emotions, thereby realizing a situation in which users can act with peace of mind during events or disasters.

[1329] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1330] Step 1: User opens dedicated application

[1331] How it works: The user launches a dedicated application on their device.

[1332] Input: User operation (application startup)

[1333] Output: The application starts and the input screen is displayed.

[1334] Step 2: User inputs and attaches the situation on site and sends it

[1335] How it works: The user enters text into the text box, attaches images and videos, for example, "Band X is currently performing on Stage B," and then presses the send button.

[1336] Input: Text data, image files, video files

[1337] Output: An information packet with automatically added location and timestamp is generated and sent to the server.

[1338] Step 3: The server receives the information

[1339] Operation: The server receives an information packet sent from the user terminal.

[1340] Input: Information packets (text, images, videos, location information, timestamps)

[1341] Output: The received information is stored in a database.

[1342] Step 4: The server filters the information

[1343] How it works: The server uses specific algorithms to filter out unwanted information from the information it receives, for example filtering out spam messages and irrelevant images.

[1344] Input: Incoming data (text, image, video, location, timestamp)

[1345] Output: Filtered and relevant information is extracted.

[1346] Step 5: Server clustering information

[1347] How it works: A generative AI in the server analyzes the filtered information and groups highly relevant information together, for example, all posts about Stage B into one cluster.

[1348] Input: Filtered information

[1349] Output: Clustered information (information within a cluster is integrated)

[1350] Step 6: Generative AI extracts keywords and generates descriptions

[1351] How it works: The generative AI extracts important keywords from the clustered information and generates a concise description based on them. For example, it might generate a description like, "Currently, band X is performing on Stage B, so it may be crowded."

[1352] Input: Clustered information

[1353] Output: Description

[1354] Step 7: The emotion engine recognizes the user's emotions

[1355] How it works: The emotion engine in the server analyzes the information sent by the user and recognizes the user's emotions, for example, determining excitement or confusion from the wording in the text and facial expressions in the images.

[1356] Input: Information sent (text, images, videos)

[1357] Output: User's emotional information

[1358] Step 8: Reflect emotions in your description

[1359] How it works: Based on the emotional information recognized by the emotion engine, the generation AI reflects emotional elements in the description. For example, it adds the expression "There is a large audience and the atmosphere is lively."

[1360] Input: Emotion information, generated description

[1361] Output: Final description reflecting the sentiment

[1362] Step 9: The server generates and sends a challenge requesting additional information.

[1363] How it works: The server examines the generated explanation, identifies missing information, and automatically generates follow-up questions. For example, it generates a question like, "Could you please tell me more about your specific situation?" and sends it to the user's device.

[1364] Input: Final description

[1365] Output: Additional questions

[1366] Step 10: User provides additional information and submits

[1367] How it works: The user responds to a follow-up question from the server by providing additional information, such as text or images, and submitting it.

[1368] Input: Additional information (text, image)

[1369] Output: Additional information is sent to the server.

[1370] Step 11: Server receives additional information and re-aggregates

[1371] How it works: The server receives additional information sent by the user and re-aggregates it with existing data.

[1372] Input: Additional Information

[1373] Output: Complete information aggregated in a database

[1374] Step 12: The generation AI regenerates the final description

[1375] How it works: The AI ​​then regenerates the final description based on the re-aggregated information. For example, it might generate a specific description like, "Currently, band X is performing on stage B, causing congestion. Band Y is preparing on stage C."

[1376] Input: Re-aggregated information

[1377] Output: Final description

[1378] Step 13: The server delivers the final description

[1379] How it works: The server distributes the final description to other users, who then display it as notifications or feeds on their devices.

[1380] Input: Final description

[1381] Output: Notifications and feeds delivered to user devices

[1382] This allows users to obtain fast and reliable information in real time, and provides emotionally sensitive feedback. This system creates an environment where users can act with confidence during events and disasters.

[1383] (Application example 2)

[1384] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1385] The rapid sharing and efficient management of information within a logistics center is extremely important for improving work efficiency and reducing errors. However, there are often delays in collecting information from the field and providing feedback based on that information, which can result in work delays and incorrect deliveries. Furthermore, there is a need to provide optimal feedback based on the emotions and urgency of field workers, but current systems have difficulty in this regard. To solve these issues, a system is needed that quickly and accurately collects field information and provides feedback that takes into account the emotions of workers.

[1386] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1387] In this invention, the server includes means for receiving information transmitted from a user terminal, means for filtering the received information to eliminate unnecessary information, means for analyzing the filtered information and clustering related information, means for extracting keywords from the clustered information and generating a concise explanation, means for analyzing the emotional state of the user using an emotion engine and reflecting the emotion in the explanation, means for identifying deficiencies in the generated explanation and transmitting additional information to the user, means for receiving the additional information and regenerating a final explanation, and means for distributing the final explanation to other users. This improves the efficiency of information sharing and management within the logistics center and enables prompt and appropriate feedback that takes into account the emotions of field workers.

[1388] A "user terminal" is a device used by workers in a logistics center to input and transmit information.

[1389] A "server" is a device that receives information sent from a user terminal, and performs filtering, analysis, clustering, description generation, sentiment analysis, sending additional information, and regenerating final information.

[1390] "Filtering" refers to the process of filtering out unnecessary information from received information and extracting only the necessary information.

[1391] "Clustering" refers to the process of grouping filtered information based on relevance.

[1392] "Keyword extraction" is the process of selecting important elements from clustered information.

[1393] "Description generation" is the process of creating concise, easy-to-understand text based on the important elements obtained through keyword extraction.

[1394] An "emotion engine" is a system that analyzes information input by a user and recognizes the user's emotional state from that information.

[1395] "Sending additional information" refers to an operation of sending a question or request to the user to fill in any gaps in the generated explanation.

[1396] "Receiving additional information" refers to an operation of receiving supplemental information sent by a user.

[1397] "Regeneration of final information" is the process of updating the description again based on additional information to make it more accurate.

[1398] "Delivery of final information" refers to an operation of notifying other users of the finally generated explanation.

[1399] "Emotion analysis" refers to the process of analyzing a user's emotional state through an emotion engine.

[1400] System Configuration

[1401] This invention is a system consisting of a user terminal, a server, a generative AI, and an emotion engine. The user terminal is used by workers in a logistics center and is responsible for inputting on-site information and sending it to the server. The server is a device that receives, processes, analyzes, recognizes emotions, and finally regenerates and distributes explanatory text.

[1402] Operation on the user device

[1403] On the user device, workers use a dedicated smartphone application to input on-site information. Input can be in a variety of formats, including text, images, and videos, and there is a function to input and send specific information, such as "the package label cannot be found in Section B." Location information and timestamps are also automatically added from the user device. This makes it possible to clearly identify when and where each piece of information occurred.

[1404] Processing on the server

[1405] The server receives information sent from the user terminal. The received information is first processed by a filtering algorithm to remove unnecessary or irrelevant information. Next, highly relevant information is clustered and managed in a consolidated manner.

[1406] Generative AI analyzes the clustered information, extracts important keywords, and generates concise descriptions that summarize the filtered information and convert it into an easy-to-understand format.

[1407] Processing with the Emotion Engine

[1408] The emotion engine in the server analyzes the information sent from the user's device and recognizes the worker's emotional state. For example, emotions such as "confusion," "excitement," and "relief" are determined through text and image analysis. This adds emotional elements to the generated explanation, enabling the system to provide more considerate feedback to the user.

[1409] Generate and submit follow-up questions

[1410] The server then carefully examines the initially generated description to identify any missing or ambiguous information. In such cases, the AI ​​takes into account the user's emotions as recognized by the emotion engine and automatically generates questions to request additional information. For example, for a confused user, it generates a thoughtful question such as, "Could you please explain the specific situation in more detail?"

[1411] Final information generation and distribution

[1412] The server receives additional information from the user and stores it in a database. The generation AI regenerates the final explanation based on the newly added information. For example, a specific explanation such as "The package label cannot be confirmed in section B. Upon checking, we found that the label was missing, so we attached a new label" is generated. The final explanation is notified to other workers and managers, enabling information sharing in real time.

[1413] Specific examples

[1414] For example:

[1415] Example 1: Information sharing in a logistics center

[1416] 1. Worker A types "I can't find the package label in Section B" into his smartphone and sends it.

[1417] 2. The server receives this information and performs filtering and clustering.

[1418] 3. The AI ​​generates the explanation "The package label cannot be confirmed in section B."

[1419] 4. The emotion engine recognizes worker A's confusion and reflects it in the explanation.

[1420] 5. A follow-up question is generated and sent to worker A: "Could you please tell me more about the specific situation?"

[1421] 6. Worker A responds, "I checked and found that the label was missing, so I attached a new label."

[1422] 7. The final explanation is sent to other workers: "Section B could not find the package label. Upon checking, we found that the label was missing, so we attached a new label."

[1423] Prompt Sentence Examples

[1424] "User Input: Package label not found in Section B."

[1425] "Generative AI model prompt: Please describe the situation on the ground."

[1426] "Sentiment analysis: puzzled"

[1427] "Final description: Problem with section B. Details: Unable to locate package label. Emotion: Confused."

[1428] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1429] Step 1:

[1430] Entering information on the user's device

[1431] A worker with a user device reports a specific problem or situation at the logistics center. The worker opens a dedicated smartphone application and inputs the situation in the form of text, images, or videos, such as "The package label cannot be found in Section B." The user device automatically adds location information and a timestamp.

[1432] Input: On-site information (text, images, videos), location information, timestamp

[1433] Output: Structured information data

[1434] Step 2:

[1435] Receiving and filtering information

[1436] The server receives the information sent from the user's device and filters it using specific algorithms to remove unnecessary or irrelevant information, such as typos and meaningless data from the text.

[1437] Input: Structured information data

[1438] Output: Filtered information data

[1439] Step 3:

[1440] Information Clustering

[1441] The server analyzes the filtered information and clusters related information, for example, all "questions related to Section B" into one cluster, thereby aggregating information for specific sections or situations.

[1442] Input: Filtered information data

[1443] Output: Clustered information data

[1444] Step 4:

[1445] Keyword extraction and description generation

[1446] The server's generation AI extracts important keywords from the clustered information and generates a concise explanation, such as "The luggage label cannot be found in section B."

[1447] Input: Clustered information data

[1448] Output: Generated description

[1449] Step 5:

[1450] Emotion recognition

[1451] The server uses an emotion engine to analyze information sent from the user's device and recognize the worker's emotional state. For example, it can determine whether the worker is in a "confused" state from text and images.

[1452] Input: Generated description, information from user terminal

[1453] Output: Recognized emotion data

[1454] Step 6:

[1455] Generate and submit follow-up questions

[1456] If the server determines that the generated description is insufficient, the AI ​​generates additional questions based on the emotion recognition results, such as "Could you please tell me more about the specific situation?"

[1457] Input: Generated description, recognized emotion data

[1458] Output: Additional questions

[1459] Step 7:

[1460] Receiving additional information from the user

[1461] The user terminal receives the query from the server, and the worker enters specific additional information and resubmits it. For example, the worker returns information such as, "After checking, I found that a label was missing, so I attached a new label."

[1462] Input: Additional Question

[1463] Output: Additional information data

[1464] Step 8:

[1465] Regenerate the final information

[1466] The server re-analyzes the generated AI based on the additional information and re-generates the final description, for example, "The package label cannot be found in section B. Upon checking, the label was found to be missing, so a new label was attached."

[1467] Input: Additional information data

[1468] Output: Final description

[1469] Step 9:

[1470] Final information distribution

[1471] The server notifies other users of the final generated explanation, for example, by distributing it in real time to other workers or administrators.

[1472] Input: Final description

[1473] Output: Notification description

[1474] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1475] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1476] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1477] [Fourth embodiment]

[1478] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1479] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1480] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1481] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1482] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1483] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1484] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1485] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1486] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1487] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1488] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1489] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1490] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1491] The system of the present invention is composed of a user terminal, a server, and a generation AI. Below, we will provide a detailed explanation of how each element operates and how the entire system functions.

[1492] Operation on the user device

[1493] Enter and submit information

[1494] When a user is at an event or disaster site, they open a dedicated application. They write and attach information about the situation at the site in the form of text, images, or videos, and then press the send button. For example, they could post something like, "Band X is currently performing on Stage B, and a large audience is gathering." At this time, location information and a timestamp are automatically added from the user's device.

[1495] Processing on the server

[1496] Receiving and filtering information

[1497] The server receives information sent from the user's device. The received information includes the content of the post, location information, timestamp, etc. After receiving the information, the server filters it using a specific algorithm to eliminate unnecessary or irrelevant information. The relevant information is then passed on to the next processing step.

[1498] Information clustering and keyword extraction

[1499] The filtered information is analyzed by a generation AI in the server, which clusters the filtered information into related groups (e.g., all information related to Stage B into one cluster).The generation AI then extracts important keywords from each cluster and generates a concise description.

[1500] Examining the generated explanation and generating follow-up questions

[1501] The server then carefully examines the generated description to identify any missing information or ambiguities. For example, if the generated description is "Stage B is currently crowded because band X is playing," the server will automatically generate additional questions about the situation on other stages and the number of audience members.

[1502] Asking the user additional questions and completing information

[1503] Submit and answer additional questions

[1504] The server generates a follow-up question and sends it to the user. For example, a question like "What is the status of the other stages?" is sent to the user. The user responds to the question by providing additional information in the form of text or images. This answer is then sent back to the server.

[1505] Final information generation and distribution

[1506] Re-aggregating information and regenerating descriptions

[1507] The server receives additional information from the user and stores it in a database. The generation AI then regenerates the final description based on the newly added information. For example, a specific description such as "Currently, band X is performing on stage B, causing congestion. Band Y is preparing on stage C." is generated.

[1508] Final information distribution

[1509] The server then distributes the resulting descriptions to other users in real time, who then display the information as notifications or feeds.

[1510] Specific examples

[1511] Music festival scenario

[1512] 1. User A posts at a music festival, "Band X is performing on Stage B."

[1513] 2. The server receives this post and clusters it with other related posts.

[1514] 3. The AI ​​generates a description like "Stage B is currently crowded because band X is performing," but it lacks information about the other stages.

[1515] 4. The server sends a follow-up question to User A: "What's the status of the other stages?"

[1516] 5. User A answers, "Band △△ is preparing for stage C."

[1517] 6. The final description will be "Currently, stage B is crowded as band X is performing. band YY is preparing on stage C."

[1518] Disaster scenarios

[1519] 1. User B posts, "The roads are cracked due to the earthquake."

[1520] 2. The server receives this post and parses it along with other related posts.

[1521] 3. The AI ​​generates the description "Roads are cracked due to the earthquake. Aftershocks are continuing," but it lacks information about evacuation sites.

[1522] 4. The server asks User B an additional question: "Where is the evacuation site?"

[1523] 5. User B answers, "The evacuation site is a nearby park."

[1524] 6. Finally, the following message is displayed: "Roads are cracked due to the earthquake. Aftershocks are continuing. Evacuation shelter is in a nearby park."

[1525] The system of the present invention allows users to obtain fast and reliable information in real time, enabling accurate situational understanding even during events or disasters.

[1526] The processing flow will be explained below.

[1527] Step 1:

[1528] The user opens the dedicated application, writes and attaches the situation at the site in the form of text, images, videos, etc., and presses the send button. Location information and a timestamp are automatically added from the user's device.

[1529] Step 2:

[1530] The server receives the information sent from the user terminal, including the posted text, images, videos, location information, and timestamps.

[1531] Step 3:

[1532] The server filters the received information using specific algorithms to remove unnecessary or irrelevant information, and stores the filtered information for analysis.

[1533] Step 4:

[1534] Based on the filtered information, the generative AI clusters this information, specifically grouping highly related information together.

[1535] Step 5:

[1536] From the clustered information, the generative AI extracts important keywords, and generates an initial, concise description based on the extracted keywords.

[1537] Step 6:

[1538] The server reviews the generated description to identify any missing or ambiguous information, confirms the additional information needed, and proceeds to the next step.

[1539] Step 7:

[1540] The server automatically generates additional questions based on the missing information and sends the generated questions to the corresponding user terminal.

[1541] Step 8:

[1542] The user terminal receives the additional question, and the user provides an answer (text, image, etc.) to the question. The user presses the send button to send the answer to the server.

[1543] Step 9:

[1544] The server receives additional information from the user and passes it back to the AI ​​for re-analysis, which then generates the final description.

[1545] Step 10:

[1546] The server then distributes the final description to other users, who then display it in real time as notifications or feeds on their devices.

[1547] This allows users to obtain fast and accurate information in real time.

[1548] Example 1

[1549] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1550] During modern events and disasters, it is important to quickly and accurately grasp the situation at the scene, but manually collecting and organizing information takes time and effort. It is also difficult to ensure the reliability of the information and real-time availability of the information. Furthermore, when efficiently processing multi-format data (text, images, and video) and converting it into easy-to-understand descriptions, additional information from users is often not reflected in a timely manner. To address these issues, effective information reception, filtering, clustering, keyword extraction, and improved accuracy of automatically generated descriptions are required.

[1551] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1552] In this invention, the server includes means for receiving multi-format data transmitted from a user terminal, means for initially processing the received data and eliminating irrelevant data, means for clustering the filtered data and organizing it into related information groups, means for extracting keywords from the clustered information groups and generating an explanatory text, means for identifying missing information in the generated explanatory text and requesting additional information from the user, means for receiving the additional information from the user and regenerating a final explanatory text, and means for distributing the final explanatory text to other users in real time, thereby enabling efficient processing of multi-format data and providing reliable information in real time.

[1553] A "user terminal" is a computing device that a user uses to input and transmit information, and includes smartphones, tablets, laptops, and the like.

[1554] "Multi-format data" refers to digital data in different formats, such as text, images, and video.

[1555] A "server" is a computer system that receives, processes, stores, and distributes data sent from user terminals over a network.

[1556] "Initial processing" refers to processing to analyze received data and eliminate signal noise and unnecessary information.

[1557] "Filtering" refers to the process of removing unnecessary information from received data and extracting highly relevant information.

[1558] "Clustering" is a technical method for grouping data that have similar characteristics.

[1559] An "information group" is a collection of a series of clustered related information.

[1560] "Keyword extraction" is the process of identifying and extracting important words and phrases from text data.

[1561] "Explanation" refers to text information generated in a format that is easy for users to understand.

[1562] A "generative AI model" is an algorithm or model that uses artificial intelligence to analyze data, extract keywords, and generate descriptions.

[1563] "Additional information" refers to information requested from the user to reacquire data that was missing at the time of initial information collection.

[1564] "Real-time delivery" is the process of providing the results to other users immediately at the moment the data is collected and analyzed.

[1565] The system of the present invention is composed of a user terminal, a server, and a generative AI model. Below, we will explain in detail how each element operates and how the entire system functions.

[1566] Hardware and Software Configuration

[1567] User device:

[1568] A user device is a computing device used by a user, and includes smartphones, tablets, laptops, etc. Using these devices, users open a dedicated application and enter information about an event or disaster site in the form of text, images, and videos. For example, a user might enter text such as "Band X is currently performing on Stage B," attach a photo, and press the send button. The user device automatically adds location information and a timestamp before sending the information.

[1569] server:

[1570] The server is a computer system that receives and processes multi-format data sent from user devices via a network. The received data is stored in a real-time database, and irrelevant and inconsistent data is eliminated as an initial process. The server then filters the data using a specific algorithm to extract only highly relevant data. The filtered data is then clustered using a generative AI model and organized into related information groups. Machine learning techniques are used for this clustering process. The generative AI model then extracts keywords from each cluster and generates a concise explanation to be communicated to the user.

[1571] Generative AI models:

[1572] The generative AI model is an algorithm that extracts keywords from the clustered information and generates explanatory text. If the generated explanation is incomplete, the server identifies this and generates and sends a prompt to the user requesting additional information.

[1573] Specific examples

[1574] Music festival scenario

[1575] 1. User A is at a music festival and types in the text "XX band is performing on Stage B," attaches a photo of the stage, and presses the send button.

[1576] 2. The server receives this information and stores the data, including location and timestamp, in a real-time database.

[1577] 3. A filtering algorithm is applied to remove unnecessary data.

[1578] 4. The generative AI model clusters related information and combines information related to "Stage B" into a single cluster.

[1579] 5. The generative AI model extracts keywords from the cluster and generates a description such as, "Stage B is crowded because band X is performing."

[1580] 6. The server generates a follow-up question: "What is the status of the other stages?" and notifies User A.

[1581] 7. User A replies, "Band △△ is being prepared on Stage C," and the information is sent again.

[1582] 8. The server receives the additional information, and the generative AI model regenerates the final description: "Currently, stage B is crowded because band X is performing. On stage C, band YY is preparing."

[1583] 9. The server distributes the final information to other users in real time.

[1584] Disaster scenarios

[1585] 1. User B posts, "The roads are cracked due to the earthquake," attaches a photo of the scene, and presses the send button.

[1586] 2. The server receives this information and stores it in a real-time database.

[1587] 3. A filtering algorithm is applied to extract only the required data.

[1588] 4. The generative AI model clusters related information and combines information about "earthquakes" into one cluster.

[1589] 5. The generative AI model extracts keywords from the cluster and generates a description such as, "The roads are cracked due to the earthquake. Aftershocks are continuing."

[1590] 6. The server generates an additional question, "Where is the evacuation site?" and notifies User B.

[1591] 7. User B replies, "The evacuation site is a nearby park," and the information is sent again.

[1592] 8. The server receives the additional information, and the generative AI model regenerates the final description: "Roads are cracked due to the earthquake. Aftershocks are continuing. Evacuation shelters are in nearby parks."

[1593] 9. The server distributes the final information to other users in real time.

[1594] This allows users to obtain fast and reliable information in real time, enabling accurate situational understanding even during events or disasters.

[1595] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1596] Step 1:

[1597] The user opens a dedicated application. The user enters information about the event or disaster site in text, image, and video format, and presses the "Send" button. At this time, the user's device automatically adds location information and a timestamp. A specific example of how this works is that the user enters "Band X is currently performing on Stage B," attaches a photo of the scene, and presses the "Send" button. The input data consists of text, images, location information, and a timestamp, and is then sent to the server.

[1598] Step 2:

[1599] The server receives multi-format data sent from user devices. The server performs initial processing of the received data and applies filtering algorithms to remove irrelevant or inconsistent data. The input data is all the information sent by the user, and the output data is clean data with unnecessary data removed. Specific operations include removing spam messages and checking format consistency.

[1600] Step 3:

[1601] The server clusters the filtered data. Using a generative AI model, related information is grouped and organized into related information clusters. The input data is the filtered data, and the output data is the clustered information clusters. Specifically, multiple posts about the same event are grouped into one cluster.

[1602] Step 4:

[1603] The server uses a generative AI model to extract keywords from the clustered information and generate a description. The input data is the clustered information, and the output data is the generated description. For example, a description such as "Stage B is crowded because band X is performing" is automatically generated.

[1604] Step 5:

[1605] The server examines the generated description and identifies any missing or ambiguous information. Based on this, the server generates a prompt to request additional information from the user. The input data is the generated description, and the output data is a prompt for additional information. Specifically, a question such as "What is the status of the other stages?" is generated.

[1606] Step 6:

[1607] The server notifies the user terminal of the generated prompt text. The user receives the notification and inputs additional information. This additional information is also sent in the form of text, images, or video. The input data is the additional information sent by the user, and the output data is additional data including the additional information. A specific operation is for the user to reply, "△△ band is being prepared on stage C," and send it.

[1608] Step 7:

[1609] The server receives additional information from the user and regenerates the final description using the generative AI model. The input data is the additional information from the user, and the output data is the regenerated final description. A specific example of how this works is that the final description might read, "Currently, band X is performing on stage B, causing congestion. Band Y is preparing on stage C."

[1610] Step 8:

[1611] The server distributes the generated final description to other users in real time. The user's device displays the final information in the form of a notification or feed. The input data is the final description, and the output data is the distributed notification. Specifically, the smartphone notification screen will display "Currently, band X is performing on stage B, causing congestion. Band Y is preparing on stage C."

[1612] This allows the system to efficiently process information from users and provide other users with reliable information in real time.

[1613] (Application example 1)

[1614] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1615] In modern society, it is extremely important to quickly and accurately collect information about abnormal situations and emergencies and notify relevant parties. However, current information collection systems often contain a lot of unnecessary information and lack real-time capabilities, making it difficult to respond quickly. Information is also often insufficient or ambiguous, making it difficult to collect additional information. This can hinder appropriate responses in emergencies. It is necessary to solve these problems and achieve efficient and rapid reporting and response to abnormal situations.

[1616] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1617] In this invention, the server includes: means for receiving information transmitted from a user terminal; means for filtering the received information to eliminate unnecessary information; means for analyzing the filtered information and clustering relevant information; means for extracting keywords from the clustered information and generating a concise explanation; means for identifying deficiencies in the generated explanation and transmitting additional information to the user; means for receiving the additional information and regenerating a final explanation; means for distributing the final explanation to other users; means for comprehensively analyzing the anomaly report information transmitted from the user and detecting special circumstances; and means for automatically generating follow-up questions based on the special circumstances and transmitting them to the user in real time. This enables the server to quickly and accurately collect and analyze anomaly information reported by users and efficiently supplement missing information. The generated information can then be instantly distributed to relevant parties, enabling a rapid response.

[1618] A "user terminal" is an electronic device operated by a user, and is a terminal device for inputting and transmitting information.

[1619] "Information" refers to all data sent from a user terminal, such as text, images, videos, location information, and timestamps.

[1620] "Means for receiving" refers to a mechanism for receiving and storing information sent from a user terminal.

[1621] A "filtering means" is a mechanism for filtering out unnecessary or irrelevant information from received information.

[1622] The "means for analyzing and clustering related information" is a method for analyzing the filtered information and grouping related information into one group.

[1623] "Means for extracting keywords and generating concise descriptions" is a technology for extracting important words and phrases from clustered information and creating concise sentences based on them.

[1624] The "means for identifying missing parts of the generated description and sending additional information to the user" is a mechanism for verifying the generated description, identifying missing information, and inquiring about that information from the user.

[1625] The "means for receiving additional information and regenerating the final description" is a method for creating a new, compiled final description based on the additional information provided by the user.

[1626] The "means for distributing the final description to other users" is a mechanism for transmitting the generated final description to other users in real time.

[1627] "Anomaly report information" refers to information reported by a user regarding an abnormality or emergency.

[1628] The "means for detecting special situations" is a method for analyzing abnormality report information and identifying abnormal situations based on specific conditions or patterns.

[1629] The "means for automatically generating follow-up questions and sending them to the user in real time" is a mechanism for generating necessary follow-up questions based on the detected abnormal situation and sending them to the user immediately.

[1630] The system of the present invention is composed of a user terminal, a server, and a generation AI. Below, we will provide a detailed explanation of how each element operates and how the entire system functions.

[1631] Operation on the user device

[1632] Enter and submit information:

[1633] Users use a dedicated application installed on their smartphones or smart glasses. When they witness an abnormal or emergency situation, they record the situation on the scene in the form of text, images, or videos, and press the send button. At this time, location information and a timestamp are automatically added. For example, a user might post, "I spotted a suspicious person. I'm attaching a photo."

[1634] Processing on the server

[1635] Receiving and filtering information:

[1636] The server receives the information sent from the user's device and uses the server's computing resources (e.g., AWS, Google Cloud Platform) and a filtering algorithm to remove unnecessary data from the received information. For example, it performs processing such as "excluding information about suspicious individuals."

[1637] Information clustering and keyword extraction:

[1638] The filtered information is analyzed by a generative AI (e.g., a GPT model) on the server. The generative AI uses a clustering algorithm to group related information together. It then extracts important keywords and generates a concise description. For example, it creates a description such as, "A suspicious person has been spotted in a specific location. We would like to report the current situation."

[1639] Examining the generated explanation and generating follow-up questions:

[1640] The server then examines the generated description to identify any missing information or ambiguity. For example, if the generated description is "A suspicious person was spotted in a specific location," the server automatically generates follow-up questions such as "What is the situation in other areas?". The server uses a generative AI model and prompts to generate follow-up questions.

[1641] Asking the user additional questions and completing information

[1642] Submit and answer additional questions:

[1643] The server generates a follow-up question and sends it to the user. For example, the user might ask, "What is the situation in other areas?" The user then provides additional information in the form of text or images. This answer is then sent back to the server.

[1644] Final information generation and distribution

[1645] Re-aggregate information and regenerate descriptions:

[1646] The server receives additional information from the user and stores it in a database. The AI ​​then regenerates the final description based on the newly added information. For example, a specific description such as "A suspicious person was spotted in a specific location. Other areas are being investigated."

[1647] Final information distribution:

[1648] The server then distributes the resulting description to other users in real time, and the user's device displays this information as a notification or feed, enabling fast and accurate information sharing.

[1649] Examples of concrete examples and prompts

[1650] Examples:

[1651] 1. A user reports that they have spotted a suspicious person in a specific location.

[1652] 2. The server receives the reports and filters and clusters the relevant information.

[1653] 3. The AI ​​generates a description that says "A suspicious person was spotted in a specific location," but lacks information about other areas.

[1654] 4. The server sends a follow-up question to the user: "What's the situation in other areas?"

[1655] 5. The user replies, "There are no abnormalities in other areas."

[1656] 6. The final description will be "Suspicious person spotted in specific location. No abnormalities in other areas."

[1657] Example prompt sentence:

[1658] A user reports a suspicious person in a specific location. Please suggest the next course of action based on the current situation on the scene.

[1659] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1660] Step 1:

[1661] Entering and sending information on the user's device

[1662] When a user witnesses an abnormal or emergency situation, they input information using a dedicated application installed on their smartphone or smart glasses. The input information is recorded in text, image, or video format and sent to a server by pressing the send button. Location information and a timestamp are automatically added.

[1663] Input: Text, images, videos, with automatically added location and timestamps

[1664] Output: Anomaly information sent to the server (text, images, videos, location information, timestamp)

[1665] Step 2:

[1666] Receiving and filtering information on the server

[1667] The server receives the anomaly information sent from the user terminal, and then uses a filtering algorithm to filter out unnecessary or irrelevant information.

[1668] Input: Anomaly information submitted by the user (text, image, video, location, timestamp)

[1669] Output: Filtered and relevant anomaly information

[1670] Step 3:

[1671] Server-based information clustering and keyword extraction

[1672] The filtered information is analyzed using generative AI (e.g., GPT model) to cluster relevant information, and then important keywords are extracted from the clustered information to generate a concise description.

[1673] Input: Filtered anomaly information

[1674] Output: Generated description (e.g. "A suspicious person was spotted at a specific location")

[1675] Step 4:

[1676] The server examines the generated explanation and generates follow-up questions

[1677] The server then examines the generated explanation to identify any missing information or ambiguity. It then automatically generates follow-up questions using a generative AI model and creates prompts to notify the user.

[1678] Input: Generated description

[1679] Output: Prompt for follow-up questions (e.g., "What about other areas?")

[1680] Step 5:

[1681] Sending and answering follow-up questions to users

[1682] The server generates a follow-up question and sends it to the user, who then provides additional information in the form of text, images, etc. The answer is then sent back to the server.

[1683] Input: Prompt for follow-up questions, additional information from the user (text, images, etc.)

[1684] Output: Additional information sent to the server

[1685] Step 6:

[1686] Server re-aggregates information and regenerates descriptions

[1687] The server receives additional information from the user and stores it in a database. The AI ​​then regenerates the final description based on the newly added information.

[1688] Input: Additional information from the user

[1689] Output: Final description (e.g. "Suspicious individuals were spotted in specific locations. No anomalies found in other areas.")

[1690] Step 7:

[1691] Final information generation and distribution

[1692] The server then distributes the resulting description to other users in real time, and the user's device displays this information as notifications or feeds.

[1693] Input: Final description

[1694] Output: Notifications or feeds delivered to other users

[1695] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1696] The system of the present invention is composed of a user terminal, a server, a generation AI, and an emotion engine. Below, we will provide a detailed explanation of how each element operates and how the entire system functions.

[1697] Operation on the user device

[1698] Enter and submit information

[1699] When a user is at an event or disaster site, they open a dedicated application. They write and attach information about the situation at the site in the form of text, images, or videos, and then press the send button. For example, they can post content such as "Band X is currently performing on Stage B, and a large audience is gathering." Location information and a timestamp are also automatically added from the user's device.

[1700] Processing on the server

[1701] Receiving and filtering information

[1702] The server receives the information sent from the user's device. The received content includes the posted text, images, videos, location information, and timestamps. After receiving the information, the server filters it using a specific algorithm to eliminate unnecessary or irrelevant information. The relevant information is then passed on to the next processing step.

[1703] Information clustering and keyword extraction

[1704] The filtered information is analyzed by a generation AI in the server, which clusters the filtered information into related groups (e.g., all information related to Stage B into one cluster).The generation AI then extracts important keywords from each cluster and generates a concise description.

[1705] Processing with the Emotion Engine

[1706] Emotion recognition

[1707] The emotion engine in the server analyzes information (text, images, videos, etc.) sent from the user's device and recognizes the user's emotions. For example, it determines whether the user is excited, confused, or relieved based on the wording in the text and facial expressions in the images.

[1708] Reflecting emotion in descriptions

[1709] Based on the user's emotional information recognized by the emotion engine, the generation AI reflects emotional elements in the explanation, adding emotional nuance to the explanation, making it possible to provide information that is easier to understand.

[1710] Generate and submit follow-up questions

[1711] Generate follow-up questions

[1712] The server then carefully examines the initially generated description to identify any missing information or ambiguities. Taking into account the user's emotions as recognized by the emotion engine, follow-up questions are automatically generated to confirm any additional information needed. For example, if the user is confused, a considerate question such as "Could you please explain the specific situation in more detail?" is generated.

[1713] Submit and answer questions

[1714] The server generates a follow-up question and sends it to the user, who then provides additional information in the form of text, images, etc. The answer is then sent back to the server.

[1715] Final information generation and distribution

[1716] Re-aggregating information and regenerating descriptions

[1717] The server receives additional information from the user and stores it in a database. The AI ​​then regenerates the final description based on the newly added information. For example, a specific description such as "Currently, band X is performing on stage B, and it's crowded. Excitement is building on the other stages as well."

[1718] Final information distribution

[1719] The server then distributes the resulting description to other users, who then display the information as notifications or feeds.

[1720] Specific examples

[1721] In the case of a music festival

[1722] 1. User A posts at a music festival, "Band X is performing on Stage B."

[1723] 2. The server receives this post and clusters it with other related posts.

[1724] 3. The AI ​​generates a description such as "Currently, band X is performing on stage B, so it is crowded."

[1725] 4. The server's emotion engine recognizes User A's excitement and reflects it in the description.

[1726] 5. User A is asked a further question and answers, "Band △△ is preparing for Stage C."

[1727] 6. The final explanation will be posted as follows: "Currently, band X is performing on Stage B, causing congestion. Band △△ is preparing on Stage C."

[1728] In case of disaster

[1729] 1. User B posts, "The roads are cracked due to the earthquake."

[1730] 2. The server receives this post and parses it along with other related posts.

[1731] 3. The AI ​​generates the following explanation: "The roads are cracked due to the earthquake. Aftershocks are continuing."

[1732] 4. The server's emotion engine recognizes User B's confusion and reflects it in the explanation.

[1733] 5. The server asks User B, "Where is the evacuation shelter?"

[1734] 6. User B answers, "The evacuation site is a nearby park."

[1735] 7. Finally, the following message is displayed: "Roads are cracked due to the earthquake. Aftershocks are continuing. Evacuation shelters are available at nearby parks."

[1736] The system of the present invention allows users to obtain fast and reliable information in real time and provides feedback that takes into account the user's emotions, thereby realizing a situation in which users can act with peace of mind during events or disasters.

[1737] The processing flow will be explained below.

[1738] Step 1:

[1739] The user opens the dedicated application, writes and attaches the situation at the site in the form of text, images, videos, etc., and presses the send button. Location information and a timestamp are automatically added from the user's device.

[1740] Step 2:

[1741] The server receives the information sent from the user terminal, including the posted text, images, videos, location information, and timestamps.

[1742] Step 3:

[1743] The server uses an algorithm to filter the information it receives, eliminating unnecessary or irrelevant information, and stores the filtered information in a database for analysis.

[1744] Step 4:

[1745] The filtered information is then analyzed by a generation AI on the server, which clusters highly relevant information and groups together information related to the same topic.

[1746] Step 5:

[1747] From the clustered information, the generative AI extracts important keywords, and generates an initial, concise description based on the extracted keywords.

[1748] Step 6:

[1749] The emotion engine analyzes posted text, images, and videos to recognize the user's emotions, such as whether they are excited, confused, or relieved.

[1750] Step 7:

[1751] Based on the user's recognized emotions, the generative AI reflects emotional elements in the description, adding emotional nuance to the description and making it easier to understand.

[1752] Step 8:

[1753] The server examines the generated explanation to identify missing or ambiguous information. The emotion engine takes the user's emotions into account and automatically generates follow-up questions to confirm any additional information needed.

[1754] Step 9:

[1755] The server generates and sends follow-up questions to the user, such as "What's the status of the other stages?" or "Could you please elaborate on the specific situation?"

[1756] Step 10:

[1757] The user terminal receives the additional question, and the user provides an answer (text, image, etc.) to the question. The user presses the send button to send the answer to the server.

[1758] Step 11:

[1759] The server receives additional information from the user and passes it to the generation AI for re-analysis, which regenerates the final description.

[1760] Step 12:

[1761] The server then distributes the resulting description to other users, who then display it in real time as notifications or feeds on their devices, allowing users to obtain fast and accurate information in real time.

[1762] This completes the system that recognizes the user's emotions and provides appropriate information.

[1763] Example 2

[1764] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1765] In modern society, there is a demand for fast and accurate information provision during events and disasters. However, providing information that takes into account the reliability, relevance, and user emotions presents challenges. Specifically, there is a need to quickly extract necessary information from a large amount of mixed information and provide feedback that takes into account the user's emotions.

[1766] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for a user to input and transmit information in the form of text, image, or video via a user terminal, means for automatically adding location information and a timestamp when information is transmitted from the user terminal, means for the server to receive information transmitted from the user terminal, means for filtering the received information using a specific algorithm and eliminating unnecessary information, means for a generation AI to analyze the filtered information and cluster related information, means for the generation AI to extract keywords from the clustered information and generate a concise explanation, means for an emotion engine in the server to recognize the user's emotions, means for the generation AI to reflect emotional elements in the explanation based on the emotion information, means for identifying deficiencies in the generated explanation and automatically generating additional questions to confirm additional information, means for the server to transmit the additional questions to the user terminal, means for a user to provide additional information and the server to receive it, means for regenerating a final explanation based on the additional information received by the server, and means for distributing the final explanation to other users. This will enable users to obtain fast and reliable information in real time during events or disasters, and also provide emotionally sensitive feedback.

[1767] "User terminal" refers to a device that a user uses to input and send information such as text, images, and videos, and includes smartphones, tablets, and computers.

[1768] "Location information" refers to geographic coordinate information obtained using the GPS function of the user's device, and indicates the specific location from which the user sent the information.

[1769] A "timestamp" is information that records the exact date and time when a user sends information, and is an important element when organizing and analyzing information later.

[1770] A "server" refers to a system that receives information sent from user terminals and performs processes such as analysis, filtering, clustering, and regeneration, and operates in cooperation with a large number of devices.

[1771] "Filtering" refers to the process of removing unnecessary or irrelevant information from received information and selecting only highly relevant information.

[1772] An "algorithm" refers to a set of computational or processing steps for solving a specific problem, and is a technique used for filtering and clustering information.

[1773] "Generative AI" refers to an artificial intelligence system that analyzes large amounts of data, extracts important keywords, and generates explanatory text.

[1774] "Clustering" refers to the process of grouping related information, and serves to bring together information that belongs to the same theme or category.

[1775] "Keyword extraction" refers to the process of extracting important words and phrases from clustered information, and is a fundamental step in generating concise descriptions.

[1776] An "emotion engine" is a system that analyzes information such as text, images, and videos sent by users and recognizes and judges their emotions.

[1777] A "concise explanation" is a summary sentence created by the generative AI based on clustered information and keywords, and provides information to users in a clear and easy-to-understand format.

[1778] The "additional question" refers to a question that is automatically generated to identify deficiencies in the generated explanation and to ask the user for additional information.

[1779] The "final description" refers to the complete information provided by the AI ​​after incorporating additional information from the user.

[1780] "Distribution" refers to the process of providing the final generated description to other users as a notification or feed, a fast and effective way to communicate information.

[1781] MODE FOR CARRYING OUT THE INVENTION

[1782] The system of the present invention is composed of a user terminal, a server, a generation AI, and an emotion engine. The operation of each element of the system and a specific embodiment of the entire system will be described below.

[1783] Operation on the user device

[1784] First, the user opens a dedicated application. The user writes and attaches information about the situation at the scene in the form of text, images, videos, etc. For example, if the user is at a music festival, they can post something like, "Band X is currently performing on Stage B, and a large audience is gathering." The user's device automatically adds location information and a timestamp to the entered data and sends it to the server.

[1785] Processing on the server

[1786] The server receives information sent from the user's device. The received content includes posted text, images, videos, location information, and timestamps. For accurate processing, the server filters the information using specific algorithms to eliminate unnecessary information. Next, the server's generation AI analyzes the filtered information and clusters related information. From the clustered information, the generation AI extracts important keywords and generates a concise description.

[1787] Processing with the Emotion Engine

[1788] The emotion engine in the server analyzes the information sent from the user's device and recognizes the user's emotions. Specifically, it determines the user's emotions based on the content of the text, facial expressions in images, and audio in videos. Based on this, the generation AI reflects emotional elements in the explanatory text. This makes it possible to provide information that is easier to understand.

[1789] Generate and submit follow-up questions

[1790] The generated explanation is then scrutinized by the server to identify any missing information or ambiguous parts. Taking into account the user's emotions recognized by the emotion engine, follow-up questions are automatically generated to confirm additional information. For example, if the user is confused, a considerate question such as "Could you please explain the specific situation in more detail?" is generated. The server then sends the generated follow-up question to the user's device. The user then provides additional information in response to the question, such as text or images, and sends it back to the server.

[1791] Final information generation and distribution

[1792] The server receives additional information from the user and stores it in a database. The generation AI regenerates the final description based on the newly added information. For example, a specific description such as "Currently, band X is performing on stage B, causing a crowd. Excitement is building on other stages as well" is generated. The server then distributes the final description to other users. The distributed information is displayed on the user's device as notifications or feeds.

[1793] Examples and prompts

[1794] Examples of music festivals

[1795] 1. User A posts at a music festival, "Band X is performing on Stage B."

[1796] 2. The server receives this post and clusters it with other related posts.

[1797] 3. The AI ​​generates a description such as "Currently, band X is performing on stage B, so it is crowded."

[1798] 4. The server's emotion engine recognizes User A's excitement and reflects it in the description.

[1799] 5. User A is asked a further question and answers, "Band △△ is preparing for Stage C."

[1800] 6. The final explanation will be posted as follows: "Currently, band X is performing on Stage B, causing congestion. Band △△ is preparing on Stage C."

[1801] Prompt Sentence Examples

[1802] "What kind of description should be generated if the user is excited at a music festival?"

[1803] Specific examples of disasters

[1804] 1. User B posts, "The roads are cracked due to the earthquake."

[1805] 2. The server receives this post and parses it along with other related posts.

[1806] 3. The AI ​​generates the following explanation: "The roads are cracked due to the earthquake. Aftershocks are continuing."

[1807] 4. The server's emotion engine recognizes User B's confusion and reflects it in the explanation.

[1808] 5. The server asks User B, "Where is the evacuation shelter?"

[1809] 6. User B answers, "The evacuation site is a nearby park."

[1810] 7. Finally, the following message is displayed: "Roads are cracked due to the earthquake. Aftershocks are continuing. Evacuation shelters are available at nearby parks."

[1811] Prompt Sentence Examples

[1812] "What kind of explanatory text should be generated when a user is confused during a disaster?"

[1813] The system of the present invention allows users to obtain fast and reliable information in real time and receive feedback that takes into account their emotions, thereby realizing a situation in which users can act with peace of mind during events or disasters.

[1814] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1815] Step 1: User opens dedicated application

[1816] How it works: The user launches a dedicated application on their device.

[1817] Input: User operation (application startup)

[1818] Output: The application starts and the input screen is displayed.

[1819] Step 2: User inputs and attaches the situation on site and sends it

[1820] How it works: The user enters text into the text box, attaches images and videos, for example, "Band X is currently performing on Stage B," and then presses the send button.

[1821] Input: Text data, image files, video files

[1822] Output: An information packet with automatically added location and timestamp is generated and sent to the server.

[1823] Step 3: The server receives the information

[1824] Operation: The server receives an information packet sent from the user terminal.

[1825] Input: Information packets (text, images, videos, location information, timestamps)

[1826] Output: The received information is stored in a database.

[1827] Step 4: The server filters the information

[1828] How it works: The server uses specific algorithms to filter out unwanted information from the information it receives, for example filtering out spam messages and irrelevant images.

[1829] Input: Incoming data (text, image, video, location, timestamp)

[1830] Output: Filtered and relevant information is extracted.

[1831] Step 5: Server clustering information

[1832] How it works: A generative AI in the server analyzes the filtered information and groups highly relevant information together, for example, all posts about Stage B into one cluster.

[1833] Input: Filtered information

[1834] Output: Clustered information (information within a cluster is integrated)

[1835] Step 6: Generative AI extracts keywords and generates descriptions

[1836] How it works: The generative AI extracts important keywords from the clustered information and generates a concise description based on them. For example, it might generate a description like, "Currently, band X is performing on Stage B, so it may be crowded."

[1837] Input: Clustered information

[1838] Output: Description

[1839] Step 7: The emotion engine recognizes the user's emotions

[1840] How it works: The emotion engine in the server analyzes the information sent by the user and recognizes the user's emotions, for example, determining excitement or confusion from the wording in the text and facial expressions in the images.

[1841] Input: Information sent (text, images, videos)

[1842] Output: User's emotional information

[1843] Step 8: Reflect emotions in your description

[1844] How it works: Based on the emotional information recognized by the emotion engine, the generation AI reflects emotional elements in the description. For example, it adds the expression "There is a large audience and the atmosphere is lively."

[1845] Input: Emotion information, generated description

[1846] Output: Final description reflecting the sentiment

[1847] Step 9: The server generates and sends a challenge requesting additional information.

[1848] How it works: The server examines the generated explanation, identifies missing information, and automatically generates follow-up questions. For example, it generates a question like, "Could you please tell me more about your specific situation?" and sends it to the user's device.

[1849] Input: Final description

[1850] Output: Additional questions

[1851] Step 10: User provides additional information and submits

[1852] How it works: The user responds to a follow-up question from the server by providing additional information, such as text or images, and submitting it.

[1853] Input: Additional information (text, image)

[1854] Output: Additional information is sent to the server.

[1855] Step 11: Server receives additional information and re-aggregates

[1856] How it works: The server receives additional information sent by the user and re-aggregates it with existing data.

[1857] Input: Additional Information

[1858] Output: Complete information aggregated in a database

[1859] Step 12: The generation AI regenerates the final description

[1860] How it works: The AI ​​then regenerates the final description based on the re-aggregated information. For example, it might generate a specific description like, "Currently, band X is performing on stage B, causing congestion. Band Y is preparing on stage C."

[1861] Input: Re-aggregated information

[1862] Output: Final description

[1863] Step 13: The server delivers the final description

[1864] How it works: The server distributes the final description to other users, who then display it as notifications or feeds on their devices.

[1865] Input: Final description

[1866] Output: Notifications and feeds delivered to user devices

[1867] This allows users to obtain fast and reliable information in real time, and provides emotionally sensitive feedback. This system creates an environment where users can act with confidence during events and disasters.

[1868] (Application example 2)

[1869] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1870] The rapid sharing and efficient management of information within a logistics center is extremely important for improving work efficiency and reducing errors. However, there are often delays in collecting information from the field and providing feedback based on that information, which can result in work delays and incorrect deliveries. Furthermore, there is a need to provide optimal feedback based on the emotions and urgency of field workers, but current systems have difficulty in this regard. To solve these issues, a system is needed that quickly and accurately collects field information and provides feedback that takes into account the emotions of workers.

[1871] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1872] In this invention, the server includes means for receiving information transmitted from a user terminal, means for filtering the received information to eliminate unnecessary information, means for analyzing the filtered information and clustering related information, means for extracting keywords from the clustered information and generating a concise explanation, means for analyzing the emotional state of the user using an emotion engine and reflecting the emotion in the explanation, means for identifying deficiencies in the generated explanation and transmitting additional information to the user, means for receiving the additional information and regenerating a final explanation, and means for distributing the final explanation to other users. This improves the efficiency of information sharing and management within the logistics center and enables prompt and appropriate feedback that takes into account the emotions of field workers.

[1873] A "user terminal" is a device used by workers in a logistics center to input and transmit information.

[1874] A "server" is a device that receives information sent from a user terminal, and performs filtering, analysis, clustering, description generation, sentiment analysis, sending additional information, and regenerating final information.

[1875] "Filtering" refers to the process of filtering out unnecessary information from received information and extracting only the necessary information.

[1876] "Clustering" refers to the process of grouping filtered information based on relevance.

[1877] "Keyword extraction" is the process of selecting important elements from clustered information.

[1878] "Description generation" is the process of creating concise, easy-to-understand text based on the important elements obtained through keyword extraction.

[1879] An "emotion engine" is a system that analyzes information input by a user and recognizes the user's emotional state from that information.

[1880] "Sending additional information" refers to an operation of sending a question or request to the user to fill in any gaps in the generated explanation.

[1881] "Receiving additional information" refers to an operation of receiving supplemental information sent by a user.

[1882] "Regeneration of final information" is the process of updating the description again based on additional information to make it more accurate.

[1883] "Delivery of final information" refers to an operation of notifying other users of the finally generated explanation.

[1884] "Emotion analysis" refers to the process of analyzing a user's emotional state through an emotion engine.

[1885] System Configuration

[1886] This invention is a system consisting of a user terminal, a server, a generative AI, and an emotion engine. The user terminal is used by workers in a logistics center and is responsible for inputting on-site information and sending it to the server. The server is a device that receives, processes, analyzes, recognizes emotions, and finally regenerates and distributes explanatory text.

[1887] Operation on the user device

[1888] On the user device, workers use a dedicated smartphone application to input on-site information. Input can be in a variety of formats, including text, images, and videos, and there is a function to input and send specific information, such as "the package label cannot be found in Section B." Location information and timestamps are also automatically added from the user device. This makes it possible to clearly identify when and where each piece of information occurred.

[1889] Processing on the server

[1890] The server receives information sent from the user terminal. The received information is first processed by a filtering algorithm to remove unnecessary or irrelevant information. Next, highly relevant information is clustered and managed in a consolidated manner.

[1891] Generative AI analyzes the clustered information, extracts important keywords, and generates concise descriptions that summarize the filtered information and convert it into an easy-to-understand format.

[1892] Processing with the Emotion Engine

[1893] The emotion engine in the server analyzes the information sent from the user's device and recognizes the worker's emotional state. For example, emotions such as "confusion," "excitement," and "relief" are determined through text and image analysis. This adds emotional elements to the generated explanation, enabling the system to provide more considerate feedback to the user.

[1894] Generate and submit follow-up questions

[1895] The server then carefully examines the initially generated description to identify any missing or ambiguous information. In such cases, the AI ​​takes into account the user's emotions as recognized by the emotion engine and automatically generates questions to request additional information. For example, for a confused user, it generates a thoughtful question such as, "Could you please explain the specific situation in more detail?"

[1896] Final information generation and distribution

[1897] The server receives additional information from the user and stores it in a database. The generation AI regenerates the final explanation based on the newly added information. For example, a specific explanation such as "The package label cannot be confirmed in section B. Upon checking, we found that the label was missing, so we attached a new label" is generated. The final explanation is notified to other workers and managers, enabling information sharing in real time.

[1898] Specific examples

[1899] For example:

[1900] Example 1: Information sharing in a logistics center

[1901] 1. Worker A types "I can't find the package label in Section B" into his smartphone and sends it.

[1902] 2. The server receives this information and performs filtering and clustering.

[1903] 3. The AI ​​generates the explanation "The package label cannot be confirmed in section B."

[1904] 4. The emotion engine recognizes worker A's confusion and reflects it in the explanation.

[1905] 5. A follow-up question is generated and sent to worker A: "Could you please tell me more about the specific situation?"

[1906] 6. Worker A responds, "I checked and found that the label was missing, so I attached a new label."

[1907] 7. The final explanation is sent to other workers: "Section B could not find the package label. Upon checking, we found that the label was missing, so we attached a new label."

[1908] Prompt Sentence Examples

[1909] "User Input: Package label not found in Section B."

[1910] "Generative AI model prompt: Please describe the situation on the ground."

[1911] "Sentiment analysis: puzzled"

[1912] "Final description: Problem with section B. Details: Unable to locate package label. Emotion: Confused."

[1913] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1914] Step 1:

[1915] Entering information on the user's device

[1916] A worker with a user device reports a specific problem or situation at the logistics center. The worker opens a dedicated smartphone application and inputs the situation in the form of text, images, or videos, such as "The package label cannot be found in Section B." The user device automatically adds location information and a timestamp.

[1917] Input: On-site information (text, images, videos), location information, timestamp

[1918] Output: Structured information data

[1919] Step 2:

[1920] Receiving and filtering information

[1921] The server receives the information sent from the user's device and filters it using specific algorithms to remove unnecessary or irrelevant information, such as typos and meaningless data from the text.

[1922] Input: Structured information data

[1923] Output: Filtered information data

[1924] Step 3:

[1925] Information Clustering

[1926] The server analyzes the filtered information and clusters related information, for example, all "questions related to Section B" into one cluster, thereby aggregating information for specific sections or situations.

[1927] Input: Filtered information data

[1928] Output: Clustered information data

[1929] Step 4:

[1930] Keyword extraction and description generation

[1931] The server's generation AI extracts important keywords from the clustered information and generates a concise explanation, such as "The luggage label cannot be found in section B."

[1932] Input: Clustered information data

[1933] Output: Generated description

[1934] Step 5:

[1935] Emotion recognition

[1936] The server uses an emotion engine to analyze information sent from the user's device and recognize the worker's emotional state. For example, it can determine whether the worker is in a "confused" state from text and images.

[1937] Input: Generated description, information from user terminal

[1938] Output: Recognized emotion data

[1939] Step 6:

[1940] Generate and submit follow-up questions

[1941] ...

Claims

1. means for receiving information transmitted from a user terminal; means for filtering the received information to remove unwanted information; means for analyzing the filtered information and clustering related information; A means for extracting keywords from the clustered information and generating a concise description; means for identifying gaps in the generated description and sending additional information to the user; means for receiving additional information and regenerating the final narrative; a means for distributing the final description to other users; A system including:

2. The system according to claim 1 , wherein the posted information from the user terminal includes location information and a timestamp.

3. The system according to claim 1, wherein the generation AI extracts keywords based on the clustered information and generates explanatory text.

Images