System

Abstract

A system is provided.SOLUTION: The system includes a means for receiving voice input, a means for converting the voice input into text data, a means for analyzing the text data and generating information to be applied, a means for converting the generated information into voice and outputting the voice, a communication means for performing voice communication with another base, an information collection means at the time of disaster, a means for analyzing the collected information and selecting applicable information, a means for acquiring current position information, and a means for periodically updating peripheral disaster information based on the current position.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In facilities and schools managed by local governments, local residents have limited access to the latest disaster information. Obtaining information is particularly difficult for those who cannot use smartphones, such as the elderly and children. Furthermore, while information needs to be collected and shared quickly during a disaster, current information systems require operation, which is inconvenient when both hands are full. There is a need for a system that can solve these issues and provide fast and accurate information during a disaster. [Means for solving the problem]

[0005] The present invention is a system that includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating appropriate information, means for converting the generated information into voice and outputting it, means for communicating with other locations through voice, means for collecting information during disasters, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, and means for periodically updating surrounding disaster information based on the current location. This enables rapid and accurate information provision to local government facilities and local residents both during normal times and during disasters. In particular, the reliability of information can be enhanced by using a generative AI model to eliminate false information and announce only important information via voice.

[0006] A "means for accepting voice input" is a device or function that provides an interface for a user to input information by voice.

[0007] The "means for converting speech input into text data" is a speech recognition system for converting speech input by a user into text data.

[0008] The "means for analyzing the text data and generating corresponding information" refers to a processing device or algorithm for analyzing the converted text data and generating appropriate responses or instructions based on the content of the text data.

[0009] The "means for converting the generated information into voice and outputting it" is a voice synthesis system that converts the generated information into voice and plays it back to the user in real time.

[0010] The "communication means for performing voice communication with other locations" is a system including a network connection and a communication protocol for enabling voice communication between different locations.

[0011] "Means of collecting information during disasters" refers to devices or systems for collecting necessary information from sources such as television, radio, and social media during disasters.

[0012] "Means for analyzing collected information and selecting applicable information" refers to an algorithm or device that analyzes collected information, uses a generative AI model to eliminate false information, and selects only reliable information.

[0013] "Means for obtaining current location information" refers to a system for obtaining the current location of a terminal using GPS or other location tracking technology.

[0014] "Means for periodically updating surrounding disaster information based on current location" refers to a system that periodically collects and updates the latest disaster information for the surrounding area using the acquired current location information.

[0015] A "generative AI model" is a system that uses artificial intelligence technology to make decisions and generate information from large amounts of data, and is particularly used to eliminate false information. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is a system including means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating information to be responded to, means for converting the generated information into voice and outputting it, means for communicating with other bases through voice, means for collecting information in the event of a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, and means for periodically updating surrounding disaster information based on the current location. This system enables the rapid and accurate provision of information between local government facilities and to local residents both during normal times and during disasters.

[0038] System Configuration

[0039] The system includes the following main modules:

[0040] 1. Smart speaker device

[0041] 2. Central Server

[0042] 3. Multimodal AI Module

[0043] Program processing

[0044] 1. Reception duties during normal times

[0045] Voice input and analysis

[0046] The user speaks the reason for their visit into the smart speaker device. The device accepts the voice input and converts the voice into text data using a speech recognition module. The converted text data is then analyzed to understand the reason for the visit.

[0047] For example, if a user says, "I'm here to see the mayor," the device will parse this text and generate directions to the appropriate department.

[0048] Response and guidance

[0049] Based on the analysis results, the terminal generates a text response and plays it back to the user in real time using speech synthesis technology.

[0050] As an example of a response, a voice saying "We will direct you to the relevant department" is played.

[0051] 2. Audio conferences during normal times

[0052] Starting an Audio Conference

[0053] The user speaks to the terminal, "Please start the audio conference." The terminal converts the speech into text and sends data to the central server instructing it to start the audio conference.

[0054] Processing on the server

[0055] The server sets up an audio conference based on the received instructions, obtains the necessary connection information, and sends it to the terminal. The terminal then prompts the necessary locations to join the audio conference.

[0056] Audio conference call

[0057] The terminals at the participating locations start an audio communication session, and the users conduct an audio conference.

[0058] 3. Disaster information dissemination during disasters

[0059] Collection of information

[0060] The server collects disaster information from television, radio, social media, etc. The collected information is analyzed using a generative AI module, and false information is filtered out.

[0061] Producing accurate information

[0062] After analysis, reliable information is generated and sent to the terminal as voice data.

[0063] Information voice announcement

[0064] The terminal announces the received disaster information to the user using voice synthesis technology.

[0065] For example, an announcement may be made such as, "An earthquake of magnitude 7 has occurred. Beware of tsunamis."

[0066] 4. Local area information announcements

[0067] Collection and transmission of location information

[0068] The device uses the built-in GPS module to obtain current location information, which is then sent to the server.

[0069] Generate surrounding information

[0070] The server collects and analyzes information about the surrounding area based on location information, and selects and generates important information, which is then sent to the device as voice data.

[0071] Regular updates and announcements

[0072] The terminal periodically announces updated information to the user using voice synthesis technology.

[0073] For example, an announcement may be made saying, "Heavy rain is causing flooding in this area. Please evacuate to higher ground."

[0074] As a result, the smart speaker system of the present invention will be able to provide prompt and accurate information to local government facilities and local residents both during normal times and during disasters. Furthermore, by using a generative AI model, the system is able to eliminate false information and provide highly reliable information.

[0075] The processing flow will be explained below.

[0076] Reception duties during normal times

[0077] Step 1:

[0078] The user speaks into the smart speaker device about the reason for their visit.

[0079] For example: "I'm here to see the mayor."

[0080] Step 2:

[0081] The device accepts voice input and uses a speech recognition module to convert the user's speech into text data.

[0082] Speech to text: "I'm here to see the mayor."

[0083] Step 3:

[0084] The device analyzes the text data and determines the appropriate response based on the reason for the visit.

[0085] Analyze reason for visit: "Visiting the Mayor"

[0086] Step 4:

[0087] The terminal generates a text response based on the results of the analysis.

[0088] Response generation: "We will direct you to the appropriate department."

[0089] Step 5:

[0090] The terminal converts the generated text response into speech and plays it back.

[0091] Text-to-speech: "We will direct you to the appropriate department."

[0092] Audio conference during normal times

[0093] Step 1:

[0094] The user speaks to the terminal, "Please start the audio conference."

[0095] Step 2:

[0096] The terminal converts the voice into text data.

[0097] Voice-to-text: "Start your audio conference."

[0098] Step 3:

[0099] The terminal sends text data to the central server to instruct the start of the audio conference.

[0100] Step 4:

[0101] The server receives the instructions, sets up the audio conference, obtains the necessary connection information, and sends it to the terminal.

[0102] Acquiring and sending destination information

[0103] Step 5:

[0104] The terminal prompts the necessary locations to join the audio conference.

[0105] Step 6:

[0106] The users hold audio conferences at the designated terminals.

[0107] Disaster information dissemination during disasters

[0108] Step 1:

[0109] The server collects disaster information from television, radio, and social media.

[0110] Step 2:

[0111] The server analyzes the collected information using a generative AI module and eliminates false information.

[0112] Step 3:

[0113] The server generates reliable and accurate information and converts it into voice data.

[0114] Step 4:

[0115] The server transmits disaster information to each terminal and instructs it to make voice announcements.

[0116] Step 5:

[0117] The terminal processes the transmitted disaster information using a voice synthesis system and announces it to the user.

[0118] Example announcement: "An earthquake of magnitude 7 has occurred. Beware of tsunamis."

[0119] Announcement of local information

[0120] Step 1:

[0121] The device obtains its current location using the built-in GPS module.

[0122] Step 2:

[0123] The terminal transmits the acquired location information to a central server.

[0124] Location: "Your current location is XXX, YYY."

[0125] Step 3:

[0126] The server collects and analyzes information about surrounding disasters based on location information.

[0127] Step 4:

[0128] The server analyzes the collected peripheral information and selects and generates important information.

[0129] Step 5:

[0130] The server transmits the generated information to each terminal and instructs it to make a voice announcement.

[0131] Step 6:

[0132] The terminal synthesizes the received information into voice and announces it to the user.

[0133] Example announcement: "Heavy rain is causing flooding in this area. Please evacuate to higher ground."

[0134] Information sharing between bases

[0135] Step 1:

[0136] The user speaks to the terminal, "Please start an emergency meeting."

[0137] Step 2:

[0138] The terminal converts the voice into text data.

[0139] Step 3:

[0140] The terminal transmits text data to the central server, instructing the start of an emergency meeting.

[0141] Step 4:

[0142] The server sets up an emergency meeting, acquires connection destination information, and transmits it to the terminal.

[0143] Step 5:

[0144] The terminal prompts necessary locations to participate in the emergency meeting.

[0145] Step 6:

[0146] The user holds an emergency meeting at the designated terminal.

[0147] In this way, the smart speaker system of the present invention can provide quick and accurate information to local government facilities and local residents both during normal times and during disasters.

[0148] Example 1

[0149] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0150] The present invention aims to provide a system that eliminates false information and provides highly reliable information in situations where rapid and accurate information provision is required during disasters. It also aims to improve business efficiency and facilitate the implementation of voice conferences during normal times.

[0151] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0152] In this invention, the server includes means for accepting voice input from a user, means for converting the voice input into text data, means for analyzing the text data and generating information to be addressed, means for converting the generated information into voice and outputting it, means for communicating with other locations by voice, means for collecting information during a disaster, means for analyzing the collected information, eliminating false information, and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, means for generating and selecting the information using a generative AI model, and means including a voice recognition module and a voice synthesis module. This enables rapid and accurate information provision, improved business efficiency, and smooth voice conferences during disasters and normal times.

[0153] A "means for accepting voice input" is a device for detecting and recording voices uttered by a user.

[0154] The "means for converting voice input into text data" refers to software or algorithms for analyzing acquired voice data and converting it into text information.

[0155] "Means for analyzing text data and generating corresponding information" refers to a system that uses converted text data to understand and analyze the user's intentions and requests, and generates appropriate responses and guidance.

[0156] The "means for converting the generated information into voice and outputting it" refers to a voice synthesis technology and its output mechanism for reproducing text information as voice.

[0157] "Communication means for performing voice communication with other locations" refers to technology and systems for performing voice communication with multiple remote locations via a network.

[0158] "Means of collecting information during disasters" refers to systems and technologies for collecting relevant information from various sources when a disaster occurs.

[0159] "Means for analyzing collected information, eliminating false information, and selecting applicable information" refers to technology that evaluates the credibility of collected information, eliminates incorrect information, and selects highly reliable information.

[0160] "Means for obtaining current location information" refers to a system that uses GPS or other location information acquisition technology to determine the device's current location.

[0161] "Means for regularly updating disaster information in the surrounding area based on the current location" refers to a system that continuously obtains information on the disaster situation in the surrounding area based on the acquired location information and provides the latest information.

[0162] "Means for utilizing generative AI models" refers to systems that utilize artificial intelligence technology to generate appropriate information from collected and analyzed data.

[0163] A "voice recognition module" is a technology and software for analyzing voice data and converting it into text data.

[0164] A "voice synthesis module" is a technology and software for converting text data into voice data and playing it back.

[0165] The present invention relates to a system that receives, converts, and analyzes voice input, and converts the generated information into voice for output. Specific ways in which the present invention can be implemented are described below.

[0166] Hardware Configuration

[0167] 1. Smart speaker device:

[0168] A microphone to accept voice input

[0169] Speakers for playing audio

[0170] GPS module for obtaining location information

[0171] 2. Central Server:

[0172] High-performance server for data processing and analysis

[0173] A network interface for communicating with smart speaker devices in various locations

[0174] 3. Communications infrastructure:

[0175] A fast and stable internet connection

[0176] Software Configuration

[0177] 1. Speech Recognition Module:

[0178] Natural language processing (NLP) techniques for converting voice input into text data (e.g., popular cloud-based speech recognition services)

[0179] 2. Generative AI Model:

[0180] Artificial intelligence techniques (e.g., generative AI models) for analyzing text data and generating appropriate responses or information

[0181] 3. Speech synthesis module:

[0182] Speech synthesis technology to convert the generated text data into speech (e.g., a cloud-based speech synthesis service)

[0183] 4. Disaster information collection and analysis system:

[0184] Software that collects and analyzes information from various sources (e.g., television, radio, social media)

[0185] Operation overview

[0186] Reception duties during normal times

[0187] 1. Voice input and analysis:

[0188] The user speaks into the smart speaker device and explains the reason for their visit, for example, "I'm here to see the mayor."

[0189] The device accepts voice input and converts the voice into text data using a voice recognition module.

[0190] The converted text data is sent to a central server for analysis to understand the reason for the visit.

[0191] 2. Responses and Guidance:

[0192] The server generates a response to the user based on the analysis results.

[0193] The device uses a speech synthesis module to play back the response aloud.

[0194] Example: "We will refer you to the appropriate department."

[0195] Audio conference during normal times

[0196] 1. Start an audio conference:

[0197] A user says, "Start audio conference."

[0198] The device converts the speech into text and sends it to a central server.

[0199] The server sets up the audio conference and sends the necessary connection information to the terminal.

[0200] 2. Audio Conference Procedures:

[0201] An audio communication session is initiated between the terminals, and the users are audio conferencing.

[0202] Disaster information dissemination during disasters

[0203] 1. Information Collection and Analysis:

[0204] The server collects disaster information from various sources and uses a generative AI model for analysis to eliminate false information.

[0205] 2. Information Generation and Announcement:

[0206] The server generates reliable information and sends it to the terminal.

[0207] The terminal uses voice synthesis technology to announce information to the user.

[0208] Example: "An earthquake of magnitude 7 has occurred. Beware of tsunamis."

[0209] Announcement of local information

[0210] 1. Obtaining and sending location information:

[0211] The device uses a GPS module to obtain its current location information and transmits it to a central server.

[0212] 2. Peripheral information generation and announcement:

[0213] The server collects and analyzes information about disasters in the surrounding area based on location information, selects important information, and sends it to the terminal.

[0214] The device uses voice synthesis technology to periodically announce information to the user.

[0215] For example: "Heavy rain is causing flooding in this area. Please evacuate to higher ground."

[0216] Specific examples (related to generative AI models)

[0217] Prompt Sentence Examples

[0218] "When a user asks a smart speaker how to respond in the event of a disaster, the system generates prompts to provide reliable information."

[0219] Specific prompt sentences

[0220] "A magnitude 7 earthquake has occurred. Please provide evacuation routes and precautions to users living in areas where a tsunami may occur."

[0221] With the above-described configuration and operation, the smart speaker system of the present invention enables prompt and accurate information provision to local government facilities and local residents both during normal times and in times of disaster.

[0222] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0223] Step 1: Accepting voice input

[0224] The user speaks into the smart speaker device. The device uses a microphone to capture voice data. The input is the user's voice data, and the output is the captured voice data. This voice data is sent to the voice recognition module in the next step.

[0225] Step 2: Convert voice to text

[0226] The terminal sends the acquired voice data to the voice recognition module, which converts the voice data into text data. The input is the acquired voice data, and the output is text data. This text data is analyzed in the next step.

[0227] Step 3: Analyzing the text data

[0228] The terminal sends the converted text data to a central server, which then analyzes the text data using a natural language processing (NLP) module to understand the user's intent. For example, the text data "I've come to see the mayor" is analyzed to determine the appropriate department to respond to the request. The input is text data, and the output is the analysis result.

[0229] Step 4: Generate response text

[0230] The server generates a response to the user based on the analysis results. A generative AI model is used for this generation. The input is the analysis results, and the output is response text data. For example, the generated text might say, "We will direct you to the relevant department."

[0231] Step 5: Respond with text-to-speech

[0232] The terminal sends the response text data received from the server to the speech synthesis module. The speech synthesis module converts the text data into voice data and plays it back to the user through a speaker. The input is the response text data, and the output is voice data. For example, the voice may say, "We will direct you to the relevant department."

[0233] Step 6: Start an audio conference

[0234] The user speaks to the terminal, "Please start the audio conference." The terminal converts the speech into text data and sends it to the central server. The input is the user's voice data, and the output is the instruction text to start the audio conference.

[0235] Step 7: Configure the server for audio conferencing

[0236] The server receives the voice conference start instruction and sets up the voice conference. It obtains the necessary connection information using VOIP technology and sends it to the terminal. The input is the voice conference start instruction text, and the output is the connection information.

[0237] Step 8: Start a voice communication session

[0238] Each terminal starts a voice communication session based on the connection destination information from the server. This allows users to hold a voice conference. The input is the connection destination information, and the output is a voice communication session.

[0239] Step 9: Collect and analyze disaster information

[0240] The server collects disaster information from sources such as television, radio, and social media. It then analyzes the collected information using a generative AI model to eliminate false information. The input is disaster information data, and the output is highly reliable disaster information.

[0241] Step 10: Disaster information generation and announcement

[0242] The server generates highly reliable information after analysis and sends it to the terminal. The terminal converts the received information into voice data using voice synthesis technology and announces it to the user. The input is highly reliable disaster information, and the output is a voice announcement of the disaster information. For example, the announcement might say, "An earthquake with a seismic intensity of 7 has occurred. Beware of tsunamis."

[0243] Step 11: Obtaining and sending location information

[0244] The device acquires its current location information using the built-in GPS module and sends it to the server. The input is the location information, and the output is the data sent to the server.

[0245] Step 12: Generate and announce surrounding information

[0246] The server collects and analyzes information about surrounding disasters based on the received location information. It selects important information and sends it to the terminal, which then uses voice synthesis technology to announce it to the user. The input is location information, and the output is a voice announcement of surrounding disaster information. For example, the announcement might say, "Heavy rain is causing flood damage in this area. Please evacuate to higher ground."

[0247] This explains the process by which the system of the present invention provides rapid and accurate information while clarifying the specific operations at each processing step.

[0248] (Application example 1)

[0249] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0250] In modern society, there is a demand for fast and accurate information in the event of a disaster. However, with conventional methods, the collection, analysis, and distribution of disaster information are not centralized, which creates the risk of erroneous information spreading. Furthermore, it is difficult to provide individual disaster information based on real-time location information. This can lead to delays in evacuation behavior and the spread of anxiety-inducing information.

[0251] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0252] In this invention, the server includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating appropriate information, means for converting the generated information into voice and outputting it, means for communicating with other bases through voice, means for collecting information during a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, means for communicating with a central server to acquire and analyze disaster information, means for associating the acquired location information with the disaster information and notifying the user of the association by voice, a communication module for acquiring data from the server, and means for providing disaster information in real time using a voice assistant, thereby enabling accurate and prompt information provision during a disaster.

[0253] A "means for accepting voice input" is a device or software for recognizing and capturing voice instructions from a user.

[0254] "Means for converting voice input into text data" refers to technology for analyzing received voice data and converting it into text information.

[0255] The "means for analyzing text data and generating corresponding information" is a function for processing the converted text data and extracting and generating the necessary information.

[0256] The "means for converting the generated information into voice and outputting it" is a system for converting the generated information into synthesized voice and letting the user hear it.

[0257] "Communication means for performing voice communication with other locations" refers to a network function for exchanging voice data between multiple locations.

[0258] The "information gathering means in the event of a disaster" is a device or system for acquiring relevant information in the event of a disaster.

[0259] "Means for analyzing collected information and selecting applicable information" refers to the function of examining collected data and selecting useful information.

[0260] "Means for obtaining current location information" refers to technology for identifying the current location of a user or device.

[0261] "Means for regularly updating information about disasters in the surrounding area based on the current location" is a function that continuously reflects the latest information about disasters in the surrounding area based on the acquired location information.

[0262] "Means for communicating with a central server to obtain and analyze disaster information" refers to a system for obtaining disaster information from a central database and reviewing and processing that information.

[0263] "Means of associating acquired location information with disaster information and notifying the user as voice" refers to a function that compares location data with disaster information and notifies the user of the generated information as voice.

[0264] A "communication module for acquiring data from a server" is a device for acquiring information from a server via a network.

[0265] "Means of providing disaster information in real time using voice assistants" is a system that uses AI technology to provide disaster information instantly via voice.

[0266] This invention is a smart assistant system that accepts voice input, analyzes it, and outputs voice. Its purpose is to provide accurate information in real time, especially during disasters. The system works in conjunction with several main modules.

[0267] A user uses a smartphone to provide voice input. The smartphone uses a microphone to capture the speech and converts it into text data using speech recognition software (e.g., the speech_recognition library). The converted text data is then analyzed (e.g., using natural language processing techniques) to generate the requested information.

[0268] Next, the smartphone sends the acquired location information (e.g., using the geopy library to acquire GPS data) to the server, which then collects disaster information from a central database (e.g., using the requests library to acquire data from an API), analyzes the collected information using a generative AI model (e.g., a natural language generation model), and selects applicable information.

[0269] The server generates reliable information after analysis and sends it to the smartphone, which receives the data and announces the information to the user as voice using speech synthesis technology (e.g., the pyttsx3 library).

[0270] As a concrete example, suppose a user speaks to their smartphone, saying, "Please tell me the disaster information for my current location." In this case, the voice input is converted into text, and "disaster information" is recognized as the trigger word. The smartphone acquires the current location information and sends it to the server. The server collects and analyzes the latest disaster information around the current location, and generates accurate information such as, "An earthquake with a seismic intensity of 6 has occurred. Please evacuate immediately." The smartphone then announces this information to the user as voice.

[0271] Using generative AI models, it is possible to filter out misinformation and provide reliable information. An example prompt is as follows:

[0272] Please create the following Python code that can handle speech recognition and speech synthesis. The code will retrieve the latest disaster information related to the user's current location from the server and announce it by voice. The input speech is in Japanese.

[0273] In this way, the present invention makes it possible to provide accurate and prompt information in the event of a disaster, thereby ensuring the safety of users.

[0274] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0275] Step 1:

[0276] The user inputs voice commands using the smartphone's microphone, including commands such as "Please tell me disaster information for my current location."

[0277] Step 2:

[0278] The device accepts voice input and converts the voice into text data using voice recognition technology (e.g., the speech_recognition library). The input here is the user's voice data, and the output is a command in text format. Specifically, the device captures the user's voice from the microphone and converts it into text.

[0279] Step 3:

[0280] The terminal analyzes the converted text data and extracts the information that needs to be handled. For example, "disaster information" is recognized as a command word. The input is text data, and the output is a command word based on the analysis results. Specifically, the terminal performs keyword extraction and command analysis of the text.

[0281] Step 4:

[0282] The device uses a GPS module to obtain current location information. The input is GPS data, and the output is the current latitude and longitude information. Specifically, the device obtains the coordinates of the current location from the GPS sensor.

[0283] Step 5:

[0284] The device sends the acquired location information to a central server. The input is location data, and the output is communication packets such as HTTP requests. Specifically, the data is sent to the server via the Internet.

[0285] Step 6:

[0286] The server collects the latest disaster information based on the received location information. The collection source is television, radio, social media APIs, etc. The input is location information, and the output is the collected disaster information. Specifically, it makes an API call and retrieves the corresponding data.

[0287] Step 7:

[0288] The server analyzes the collected disaster information using a generative AI model, filtering out false information and selecting reliable information. The input is raw data, and the output is reliable information as the result of analysis. Specifically, the server performs data analysis and filtering using an AI model.

[0289] Step 8:

[0290] The server sends the analyzed information to the terminal. The input is the analysis result data, and the output is a communication packet. Specifically, the analyzed data is sent as an HTTP response.

[0291] Step 9:

[0292] The device receives the information from the server and announces it to the user as voice using speech synthesis technology (e.g., the pyttsx3 library). The input is the analysis result data, and the output is voice data. Specifically, the device converts text into voice and outputs it from the speaker.

[0293] Step 10:

[0294] The user receives voice information provided by the device and takes appropriate action in the event of a disaster. The input is voice data, and the output is the user's actions. Specifically, the user takes evacuation action by following the voice instructions from the device.

[0295] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0296] The present invention is a system that includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating information to be responded to, means for converting the generated information into voice and outputting it, means for communicating with other bases through voice, means for collecting information in the event of a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating information about disasters in the surrounding area based on the current location, and an emotion engine that recognizes the user's emotions. This system enables the rapid and accurate provision of information between local government facilities and to local residents both during normal times and during disasters.

[0297] System Configuration

[0298] The system includes the following main modules:

[0299] 1. Smart speaker device

[0300] 2. Central Server

[0301] 3. Multimodal AI Module

[0302] 4. Emotion Engine

[0303] Program processing

[0304] 1. Reception duties during normal times

[0305] Voice input and analysis

[0306] The user speaks the reason for their visit into the smart speaker device. The device accepts the voice input and converts the voice into text data using a speech recognition module. The converted text data is then analyzed to understand the reason for the visit.

[0307] emotion recognition

[0308] The emotion engine analyzes the user's voice data and recognizes the user's emotional state.

[0309] Response and guidance

[0310] Based on the analysis and emotion recognition results, the device generates a text response, adjusting the content and tone of the response depending on the results of the emotion engine.

[0311] For example, if the user is nervous, the response may be provided in a relaxing tone.

[0312] Example response: "We will direct you to the appropriate department."

[0313] Speech synthesis

[0314] The generated text response is converted into speech and played to the user in real time.

[0315] 2. Audio conferences during normal times

[0316] Starting an Audio Conference

[0317] The user speaks to the terminal, "Please start the audio conference." The voice is converted into text data, and the data instructing the central server to start the audio conference is sent.

[0318] Processing on the server

[0319] The server sets up an audio conference based on the instructions, obtains the necessary connection information, and sends it to the terminal. The terminal then prompts the necessary locations to join the audio conference.

[0320] Audio conference call

[0321] The terminals at the participating locations start an audio communication session, and the users conduct an audio conference.

[0322] 3. Disaster information dissemination during disasters

[0323] Collection of information

[0324] The server collects disaster information from television, radio, social media, etc. The collected information is analyzed using a generative AI module to eliminate false information.

[0325] Producing accurate information

[0326] After analysis, reliable information is generated and sent to the terminal as voice data.

[0327] Information voice announcement

[0328] The device processes the received disaster information using a voice synthesis system and announces it to the user.

[0329] For example, an announcement may be made such as, "An earthquake of magnitude 7 has occurred. Beware of tsunamis."

[0330] Announcement adjustment by emotion recognition

[0331] emotion recognition

[0332] The emotion engine recognizes the user's emotional state and assesses the user's stress level, especially in times of disaster.

[0333] Response Adjustment

[0334] Based on the results of the emotion engine, the content of the voice notification is adjusted to provide information in a tone and content that reduces the user's stress.

[0335] For example, if a user is in a panic, the announcement will be something like, "Please stay calm. We will provide you with the current information."

[0336] Announcement of local information

[0337] Collection and transmission of location information

[0338] The device uses its built-in GPS module to obtain its current location and transmit it to a central server.

[0339] Generate surrounding information

[0340] The server collects and analyzes information about the surrounding area based on location information, and selects and generates important information. The generated information is then sent to the device as voice data.

[0341] Regular updates and announcements

[0342] The terminal periodically announces updated information to the user using voice synthesis technology.

[0343] For example, an announcement may be made such as, "Heavy rain is causing flooding in this area. Please evacuate to higher ground."

[0344] Information sharing between bases

[0345] Emergency meeting begins

[0346] The user speaks to the terminal, "Please start an emergency meeting." The voice is converted into text data, and the data instructing the central server to start an emergency meeting is sent.

[0347] Processing on the server

[0348] The server sets up an emergency meeting, obtains the necessary connection information, and sends it to the terminal. The terminal then prompts the necessary locations to join the emergency meeting.

[0349] Emergency meeting proceedings

[0350] The terminals at the participating locations start a voice communication session, and the users hold an emergency conference.

[0351] As a result, the smart speaker system of the present invention can provide prompt and accurate information to local government facilities and local residents both during normal times and during disasters. Furthermore, the introduction of an emotion engine enables flexible responses according to the user's psychological state, helping to reduce panic, especially during disasters.

[0352] The processing flow will be explained below.

[0353] Reception duties during normal times

[0354] Step 1:

[0355] The user speaks into the smart speaker device about the reason for their visit.

[0356] For example: "I'm here to see the mayor."

[0357] Step 2:

[0358] The device accepts voice input and uses a speech recognition module to convert the user's speech into text data.

[0359] Speech to text: "I'm here to see the mayor."

[0360] Step 3:

[0361] The device analyzes the text data, determines the reason for the visit, and determines the appropriate response.

[0362] Analyze reason for visit: "Visiting the Mayor"

[0363] Step 4:

[0364] The device simultaneously sends the voice data to an emotion engine to recognize the user's emotions, for example, whether they are nervous.

[0365] Step 5:

[0366] The device generates a text response based on the analysis and emotion recognition results, adjusting the content and tone of the response depending on the results of the emotion engine.

[0367] Produced response: "I'll connect you to the right person." Plus, a relaxing tone

[0368] Step 6:

[0369] The terminal converts the generated text response into speech and plays it to the user in real time.

[0370] Audio conference during normal times

[0371] Step 1:

[0372] The user speaks to the terminal, "Please start the audio conference."

[0373] Step 2:

[0374] The terminal converts the voice into text data.

[0375] Voice-to-text: "Start your audio conference."

[0376] Step 3:

[0377] The terminal sends text data to the central server to instruct the start of the audio conference.

[0378] Step 4:

[0379] The server receives the instructions, sets up the audio conference, obtains the necessary connection information, and sends it to the terminal.

[0380] Acquiring and sending destination information

[0381] Step 5:

[0382] The terminal prompts the necessary locations to join the audio conference.

[0383] Step 6:

[0384] The terminal sends the user's voice to an emotion engine to recognize emotions during the meeting.

[0385] Step 7:

[0386] The server provides information to support the progress of the conference based on the results of the emotion engine, as needed.

[0387] Step 8:

[0388] The users hold audio conferences at the designated terminals.

[0389] Disaster information dissemination during disasters

[0390] Step 1:

[0391] The server collects disaster information from television, radio, and social media.

[0392] Step 2:

[0393] The server analyzes the collected information using a generative AI module and eliminates false information.

[0394] Step 3:

[0395] The server generates reliable and accurate information, converts it into voice data, and transmits it to the terminal.

[0396] Step 4:

[0397] The server transmits disaster information to each terminal and instructs it to make voice announcements.

[0398] Step 5:

[0399] The device processes the transmitted disaster information using a voice synthesis system and checks the user's emotional state using an emotion engine.

[0400] Step 6:

[0401] The terminal adjusts the announcement depending on the user's emotional state.

[0402] Example: "There has been a magnitude 7 earthquake. Beware of tsunamis." (Calm tone)

[0403] Step 7:

[0404] The user receives disaster information from the terminal.

[0405] Announcement of local information

[0406] Step 1:

[0407] The device obtains its current location using the built-in GPS module.

[0408] Step 2:

[0409] The terminal transmits the acquired location information to a central server.

[0410] Location: "Your current location is XXX, YYY."

[0411] Step 3:

[0412] The server collects and analyzes information about surrounding disasters based on location information.

[0413] Step 4:

[0414] The server analyzes the collected peripheral information and selects and generates important information.

[0415] Step 5:

[0416] The server transmits the generated information to each terminal and instructs it to make a voice announcement.

[0417] Step 6:

[0418] The terminal synthesizes the received information into voice and uses an emotion engine to recognize the user's emotional state.

[0419] Step 7:

[0420] The terminal adjusts the announcements according to the emotional state and provides information to the user.

[0421] For example: "Heavy rainfall is causing flooding in this area. Please seek refuge on higher ground." (Reassuring tone)

[0422] Information sharing between bases

[0423] Schedule an emergency meeting

[0424] Step 1:

[0425] The user speaks to the terminal, "Please start an emergency meeting."

[0426] Step 2:

[0427] The terminal converts the voice into text data.

[0428] Step 3:

[0429] The terminal transmits text data to the central server, instructing the start of an emergency meeting.

[0430] Step 4:

[0431] The server sets up an emergency meeting, obtains the necessary connection information, and transmits it to the terminal.

[0432] Step 5:

[0433] The terminal prompts necessary locations to participate in the emergency meeting.

[0434] Step 6:

[0435] The terminal sends the user's voice to an emotion engine to recognize emotions during an emergency meeting.

[0436] Step 7:

[0437] The server provides information to assist in the progress of the emergency meeting, if necessary, based on the results of the emotion engine.

[0438] Step 8:

[0439] The user holds an emergency meeting at the designated terminal.

[0440] In this way, the smart speaker system of the present invention can provide prompt and accurate information to local government facilities and local residents both during normal times and during disasters.Furthermore, the introduction of an emotion engine makes it possible to respond flexibly to the user's emotional state, which is a distinctive feature of the system, reducing the user's psychological burden.

[0441] Example 2

[0442] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0443] While systems using conventional voice recognition technology are required to provide accurate and prompt emergency information during disasters, in reality, they face challenges in effectively communicating information due to the inability to properly assess the user's stress level and the inclusion of false information. Furthermore, in normal reception work and voice conferences, it is difficult to respond according to the user's emotional state, leading to a demand for improved user experience.

[0444] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating information to be handled, means for converting the generated information into voice and outputting it, communication means for performing voice communication with other bases, means for collecting information in the event of a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, and means for recognizing the user's emotional state and adjusting the response content. This makes it possible to quickly provide accurate and appropriate information in the event of a disaster, and to flexibly respond to the user's emotional state in normal times.

[0445] The "means for accepting voice input" is a device that has the function of capturing the voice spoken by the user as a digital signal.

[0446] "Means for converting voice input into text data" refers to technology for analyzing voice signals and converting them into corresponding text data.

[0447] "Means for analyzing text data and generating corresponding information" refers to technology that processes acquired text data and extracts and generates information based on the user's requests and intentions.

[0448] The "means for converting the generated information into voice and outputting it" is a device that has the function of converting text data into a voice signal using voice synthesis technology and playing it aloud to the user.

[0449] "Communication means for performing voice communication with other locations" refers to communication technology for transmitting and receiving voice data between multiple terminals and servers.

[0450] "Means of collecting information during disasters" refers to technology that has the function of collecting disaster-related information from various sources such as television, radio, and social media.

[0451] "Means for analyzing collected information and selecting applicable information" refers to the technology of processing collected information using data analysis technology and selecting highly reliable information.

[0452] "Means for obtaining current location information" refers to technology for identifying the current location using a GPS module or the like.

[0453] "Means for periodically updating surrounding disaster information based on the current location" refers to technology that continuously collects disaster information for the surrounding area based on the acquired location information and updates it to the latest information.

[0454] "Means for recognizing the user's emotional state and adjusting the response content" refers to technology that analyzes emotions from the user's voice and adjusts the response content and tone based on the results.

[0455] MODE FOR CARRYING OUT THE INVENTION

[0456] The present invention includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating corresponding information, means for converting the generated information into voice and outputting it, communication means for performing voice communication with other bases, means for collecting information in the event of a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, and means for recognizing the user's emotional state and adjusting the response content.

[0457] The system of the present invention includes the following main modules:

[0458] 1. A means of accepting voice input (smart speaker device)

[0459] Smart speaker devices have built-in microphones that accept voice input from users. For example, if you say, "I want to go to the government office," the voice is captured by the device as a digital signal.

[0460] 2. A means of converting voice input into text data

[0461] The device converts the received voice input into text using a speech recognition module such as Google Cloud Speech-to-Text, which analyzes the voice signal and converts it into text.

[0462] 3. A method for analyzing text data and generating relevant information

[0463] The converted text data is analyzed using natural language processing (NLP) technology to extract information based on the user's intent and requests. For example, if the user enters "I want to go to the government office," the system identifies "visiting the government office" as the reason for the visit.

[0464] 4. A means of converting the generated information into audio and outputting it

[0465] The text response generated based on the analysis results is converted into speech using a speech synthesis system such as Amazon Polly. The device then plays this speech back to the user, responding, for example, in a relaxing tone, saying, "We will direct you to the appropriate department."

[0466] 5. Communication means for voice communication with other locations

[0467] To enable voice communication between the device and other locations, a connection is set up using the Zoom API for each location, allowing users to hold voice conferences and communicate through the device.

[0468] 6. Information gathering methods during disasters

[0469] The server collects disaster information from sources such as television, radio, and social media. The collected information is then analyzed using OpenAI's generative AI model to filter out unreliable information. This process ensures accurate and prompt information provision.

[0470] 7. How we analyze the collected information and select applicable information

[0471] The collected disaster information is processed using data analysis technology to extract applicable and reliable information, which is then generated in text format and transmitted to the device as voice data.

[0472] 8. How to get current location information

[0473] The device uses its built-in GPS module to determine its current location and sends it to the server using the Google Maps API.

[0474] 9. A way to periodically update information about nearby disasters based on your current location

[0475] The server collects and analyzes disaster information from the surrounding area based on the acquired location information, and selects important information. This information is updated periodically and sent to the device as voice data.

[0476] 10. A way to recognize the user's emotional state and tailor responses

[0477] The emotion engine analyzes the user's voice data and recognizes the user's emotional state using tools such as IBM Watson Tone Analyzer. Particularly in times of disaster, the system assesses the user's stress level and panic and adjusts the content and tone of the response accordingly. For example, the response may be something like, "Please stay calm. We'll provide you with the latest information."

[0478] Examples of concrete examples and prompts

[0479] For example, if a user says to a smart speaker, "I want to go to the government office," the device converts the voice into text data and analyzes the reason for the visit. If the emotion engine determines that the user is nervous, it responds in a relaxing tone, saying, "I will guide you to the relevant department," and plays the response aloud.

[0480] An example prompt is:

[0481] The system converts what the user says into text and analyzes that text to understand the reason for the visit. It then analyzes the user's emotional state and responds by voice, adjusting the tone of the response as needed. For example, if the user says, "I'd like to go to the government office," the system responds in a relaxing tone, saying, "I'll take you to the appropriate department."

[0482] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0483] Processing Steps

[0484] Step 1: Accepting voice input

[0485] Subject: Terminal

[0486] Specific behavior:

[0487] The user speaks into the smart speaker device and says, "I would like to go to the city hall."

[0488] The device captures audio using its built-in microphone.

[0489] input:

[0490] User Voice

[0491] Data processing:

[0492] Converts audio signals into digital data.

[0493] output:

[0494] Digital audio data

[0495] Step 2: Convert audio data to text

[0496] Subject: Terminal

[0497] Specific behavior:

[0498] The device converts the voice data into text data using the Google Cloud Speech-to-Text module.

[0499] input:

[0500] Digital audio data

[0501] Data processing:

[0502] The voice signal is analyzed and converted into corresponding text data.

[0503] output:

[0504] Converted text data (e.g., "I would like to go to the government office.")

[0505] Step 3: Analyzing the text data

[0506] Subject: Terminal

[0507] Specific behavior:

[0508] The device uses natural language processing (NLP) technology to analyze the text data and understand the reason for the visit.

[0509] input:

[0510] Converted text data

[0511] Data processing:

[0512] Through text data analysis, the reason for the visit is identified (e.g., "visiting government office").

[0513] output:

[0514] Analyzed information (e.g., "visit to government office")

[0515] Step 4: Emotion Recognition

[0516] Subject: Terminal

[0517] Specific behavior:

[0518] The emotion engine uses IBM Watson Tone Analyzer to analyze the user's voice data and recognize their emotional state.

[0519] input:

[0520] Converted text data

[0521] Captured audio data

[0522] Data processing:

[0523] Perform tone and sentiment analysis on voice and text data.

[0524] output:

[0525] Emotional state analysis results (e.g., tension)

[0526] Step 5: Response generation and adjustment

[0527] Subject: Terminal

[0528] Specific behavior:

[0529] Based on the analysis and emotion recognition results, the device generates a text response.

[0530] Adjust the content and tone of your response depending on the results of the emotion engine.

[0531] input:

[0532] Analyzed information (e.g., "visit to government office")

[0533] Emotional state analysis results (e.g., tension)

[0534] Data processing:

[0535] Generate text responses and adjust tone

[0536] output:

[0537] Tailored text responses (e.g., "We'll direct you to the appropriate department")

[0538] Step 6: Speech synthesis and response playback

[0539] Subject: Terminal

[0540] Specific behavior:

[0541] The device converts the generated text response into speech using a speech synthesis system such as Amazon Polly.

[0542] Plays a sound to the user.

[0543] input:

[0544] Tailored text responses (e.g., "We'll direct you to the appropriate department")

[0545] Data processing:

[0546] Convert text data into audio signals.

[0547] output:

[0548] Voice response (e.g., "We will direct you to the appropriate department")

[0549] Step 7: Collect disaster information

[0550] Subject: Server

[0551] Specific behavior:

[0552] The server collects disaster information from television, radio, social media, etc.

[0553] input:

[0554] Disaster-related information data from various sources

[0555] Data processing:

[0556] Disaster information collection and data conversion

[0557] output:

[0558] Collected disaster information data

[0559] Step 8: Analysis and selection of disaster information

[0560] Subject: Server

[0561] Specific behavior:

[0562] The server uses OpenAI's generative AI model to analyze the collected disaster information and eliminate false information.

[0563] Select reliable disaster information.

[0564] input:

[0565] Collected disaster information data

[0566] Data processing:

[0567] Data analysis and information selection

[0568] output:

[0569] Reliable disaster information

[0570] Step 9: Obtaining and sending location information

[0571] Subject: Terminal

[0572] Specific behavior:

[0573] The device uses the built-in GPS module to obtain its current location.

[0574] The acquired location information is sent to the server.

[0575] input:

[0576] Location data from the GPS module

[0577] Data processing:

[0578] Acquiring location information and improving data formats

[0579] output:

[0580] Current location information

[0581] Step 10: Collect and generate information on surrounding disasters

[0582] Subject: Server

[0583] Specific behavior:

[0584] The server collects and analyzes information about surrounding disasters based on location information.

[0585] input:

[0586] Current location information

[0587] Data processing:

[0588] Collection and analysis of information on nearby disasters

[0589] output:

[0590] Generated disaster information data

[0591] Step 11: Disaster Information Updates and Announcements

[0592] Subject: Terminal

[0593] Specific behavior:

[0594] The disaster information received by the device is converted into speech using a speech synthesis system such as Amazon Polly.

[0595] Users will be notified of the latest disaster information on a regular basis.

[0596] input:

[0597] Generated disaster information data

[0598] Data processing:

[0599] Audio data generation and announcement of disaster information

[0600] output:

[0601] Voice announcements (e.g., "Heavy rain is causing flooding in this area. Please evacuate to higher ground.")

[0602] Step 12: Recognize the user's emotional state and adjust your response

[0603] Subject: Terminal

[0604] Specific behavior:

[0605] The emotion engine analyzes the user's voice data and assesses their emotional state.

[0606] Tailor your response and tone based on the user's stress level or panic.

[0607] input:

[0608] User voice data

[0609] Emotional state analysis results

[0610] Data processing:

[0611] Emotional state assessment and response adjustment

[0612] output:

[0613] Tailored voice responses (e.g., "Please remain calm. We'll provide you with the latest information.")

[0614] As a result, the present invention enables accurate and prompt provision of information in the event of a disaster, and realizes flexible responses according to the user's emotional state during normal times.

[0615] (Application example 2)

[0616] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0617] Conventional disaster information systems have problems with insufficient real-time information provision and inability to provide information that takes into account the user's emotional state. This makes it difficult for users to take appropriate action in an emergency, increasing the likelihood of them falling into a state of panic. Furthermore, the provision of individual disaster information based on location information is insufficient, making it impossible to encourage appropriate responses based on information specific to the area. It is desirable to solve these problems and ensure the safety of users.

[0618] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating information to be addressed, means for converting the generated information into voice and outputting it, means for communicating with other locations via voice, means for collecting information during a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, emotion analysis means for recognizing emotions, and means for adjusting the content and tone of a voice response based on the emotional state recognized by the emotion analysis means. This enables accurate information to be provided in real time during disasters and emergencies, enabling flexible responses according to the user's emotional state. Furthermore, based on individual location information, disaster information specific to the region can be provided, encouraging appropriate action.

[0619] "Means for accepting voice input" refers to a device or software that has the function of detecting voice and capturing it as a digital signal.

[0620] A "means for converting speech input into text data" is a system or algorithm that analyzes received speech signals and converts them into corresponding text information.

[0621] The "means for analyzing text data and generating corresponding information" refers to a process or module that analyzes input character information and generates information to be provided to the user based on the content of that information.

[0622] The "means for converting generated information into audio and outputting it" is a technology for converting information generated on a text basis into audio and providing it to the user as audio.

[0623] "Communication means for performing voice communication with other locations" refers to communication technology and equipment for transmitting and receiving voice data between devices and systems located in different locations.

[0624] "Means of collecting information during disasters" refers to mechanisms and means for collecting useful information from outside when a disaster occurs.

[0625] "Means for analyzing collected information and selecting applicable information" refers to a system or algorithm for analyzing the various collected information and selecting information that is reliable and useful to the user.

[0626] "Means for obtaining current location information" refers to technology for measuring and obtaining the current location of the device (e.g., GPS module).

[0627] "Means for periodically updating surrounding disaster information based on current location" is a mechanism that uses acquired location information to periodically acquire and update surrounding disaster information.

[0628] "Emotion analysis means for recognizing emotions" refers to technologies and algorithms that analyze a user's voice or text and evaluate and recognize their emotional state.

[0629] The "means for adjusting the content and tone of the voice response based on the emotional state recognized by the emotion analysis means" is a function for appropriately adjusting the content and tone of the generated voice response according to the recognized emotional state of the user.

[0630] The present invention is a system for providing information during disasters and emergencies, and is composed of the following components: First, a smartphone, smart speaker, or other voice-enabled device is used as a means for users to input information by voice, allowing users to request information in real time.

[0631] To convert voice input into text data, we use the Google Speech-to-Text API. This API quickly converts voice data into text data. The text data is then analyzed and the corresponding information is generated. The software used here is the NLTK library, a Python natural language processing library. This library is used to analyze the user's input and generate the corresponding information.

[0632] The generated text data is converted into audio and provided to the user. Using the Google Text-to-Speech API, the generated text data is converted into high-quality audio. The audio output is provided through the speaker of a smartphone or smart speaker.

[0633] This system also has a communication means for voice communication with other locations. For example, when a user starts an audio conference, they can connect to other locations through a calling service such as Google Meet or Zoom and conduct voice communication.

[0634] In the event of a disaster, information is collected from television, radio, social media, etc., and analyzed to select applicable information. A generative AI model is used to analyze the collected information. This model eliminates false information and provides reliable information.

[0635] To obtain current location information, the built-in GPS module of the smartphone is used. Based on the obtained location information, information on disasters in the surrounding area is periodically updated and provided to the user. For example, heavy rain warnings and earthquake information for the area can be collected from the location information and notified to the user.

[0636] The IBM Watson Tone Analyzer is used to analyze emotions. This analyzes the user's voice and text data to recognize their emotional state. Based on the recognized emotional state, the content and tone of the voice response are adjusted. For example, if the user is nervous, information is provided in a slower, calmer tone.

[0637] Specific examples

[0638] A specific example is shown below. When a user says, "Please tell me the latest disaster information for this area," this speech is accepted as voice input and converted into text data using the Google Speech-to-Text API. The text data is then analyzed using the NLTK library, and the relevant disaster information is generated. The generated information is then converted into audio using the Google Text-to-Speech API and provided to the user in real time.

[0639] Example prompt sentence:

[0640] User says: "What are the latest disaster updates for this area?"

[0641] App response: Obtains location information, analyzes the user's emotional state with an emotion engine, and outputs collected disaster information in voice with adjusted tone.

[0642] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0643] Step 1:

[0644] The user inputs information by voice. The user speaks to a smartphone or smart speaker, saying something like, "Please tell me the latest disaster information for this area." The input data is voice data.

[0645] Step 2:

[0646] The device accepts voice input, records the audio as a digital signal, and then uses the Google Speech-to-Text API to convert this audio data into text data. The input is audio data, and the output is audio as text data.

[0647] Step 3:

[0648] The server receives the text data and analyzes it using Python's NLTK library. This analysis extracts important keywords and phrases from the input text. The data processing performed here involves analyzing the text and extracting the necessary information. The output is the analyzed text data.

[0649] Step 4:

[0650] The server generates appropriate disaster information based on the extracted keywords and phrases. Disaster information is collected using, for example, a disaster prevention information API. The generated information is expressed in text format. The input is analyzed text data, and the output is disaster information text data.

[0651] Step 5:

[0652] The server uses a generative AI model to analyze the collected disaster information and select applicable information. The generative AI model evaluates the reliability of the collected information and filters out unnecessary information. The input is text data of disaster information, and the output is reliable disaster information.

[0653] Step 6:

[0654] The server converts text data into audio data using the Google Text-to-Speech API to output the selected disaster information as audio. The input is reliable disaster information text data, and the output is audio data.

[0655] Step 7:

[0656] The voice data is provided to the user through the speaker of a smartphone or smart speaker. Conveniently, the generated voice is delivered to the user in real time. The output voice is then listened to by the user.

[0657] Step 8:

[0658] As a means of emotion analysis, IBM Watson Tone Analyzer is used to analyze the user's voice or text data to recognize their emotional state. The input is the user's voice or text data, and the output is evaluation data of the user's emotional state.

[0659] Step 9:

[0660] The server adjusts the content and tone of the voice response based on the emotion analysis data. The speed and tone of the voice change depending on the user's emotional state, reducing stress and panic in the user. The input is emotion analysis data, and the output is adjusted voice data.

[0661] For example:

[0662] When a user voice-inputs "I want disaster information," the speech is converted into text data by the Google Speech-to-Text API and analyzed by the Python NLTK library. The latest disaster information obtained from the disaster prevention information API is then converted into audio data by the Google Text-to-Speech API and provided to the user through the smartphone speaker. At the same time, the IBM Watson Tone Analyzer evaluates the user's emotional state and provides information in an appropriate tone.

[0663] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0664] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0665] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0666] [Second embodiment]

[0667] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0668] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0669] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0670] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0671] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0672] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0673] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0674] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0675] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0676] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0677] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0678] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0679] The present invention is a system including means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating information to be responded to, means for converting the generated information into voice and outputting it, means for communicating with other bases through voice, means for collecting information in the event of a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, and means for periodically updating surrounding disaster information based on the current location. This system enables the rapid and accurate provision of information between local government facilities and to local residents both during normal times and during disasters.

[0680] System Configuration

[0681] The system includes the following main modules:

[0682] 1. Smart speaker device

[0683] 2. Central Server

[0684] 3. Multimodal AI Module

[0685] Program processing

[0686] 1. Reception duties during normal times

[0687] Voice input and analysis

[0688] The user speaks the reason for their visit into the smart speaker device. The device accepts the voice input and converts the voice into text data using a speech recognition module. The converted text data is then analyzed to understand the reason for the visit.

[0689] For example, if a user says, "I'm here to see the mayor," the device will parse this text and generate directions to the appropriate department.

[0690] Response and guidance

[0691] Based on the analysis results, the terminal generates a text response and plays it back to the user in real time using speech synthesis technology.

[0692] As an example of a response, a voice saying "We will direct you to the relevant department" is played.

[0693] 2. Audio conferences during normal times

[0694] Starting an Audio Conference

[0695] The user speaks to the terminal, "Please start the audio conference." The terminal converts the speech into text and sends data to the central server instructing it to start the audio conference.

[0696] Processing on the server

[0697] The server sets up an audio conference based on the received instructions, obtains the necessary connection information, and sends it to the terminal. The terminal then prompts the necessary locations to join the audio conference.

[0698] Audio conference call

[0699] The terminals at the participating locations start an audio communication session, and the users conduct an audio conference.

[0700] 3. Disaster information dissemination during disasters

[0701] Collection of information

[0702] The server collects disaster information from television, radio, social media, etc. The collected information is analyzed using a generative AI module, and false information is filtered out.

[0703] Producing accurate information

[0704] After analysis, reliable information is generated and sent to the terminal as voice data.

[0705] Information voice announcement

[0706] The terminal announces the received disaster information to the user using voice synthesis technology.

[0707] For example, an announcement may be made such as, "An earthquake of magnitude 7 has occurred. Beware of tsunamis."

[0708] 4. Local area information announcements

[0709] Collection and transmission of location information

[0710] The device uses the built-in GPS module to obtain current location information, which is then sent to the server.

[0711] Generate surrounding information

[0712] The server collects and analyzes information about the surrounding area based on location information, and selects and generates important information, which is then sent to the device as voice data.

[0713] Regular updates and announcements

[0714] The terminal periodically announces updated information to the user using voice synthesis technology.

[0715] For example, an announcement may be made saying, "Heavy rain is causing flooding in this area. Please evacuate to higher ground."

[0716] As a result, the smart speaker system of the present invention will be able to provide prompt and accurate information to local government facilities and local residents both during normal times and during disasters. Furthermore, by using a generative AI model, the system is able to eliminate false information and provide highly reliable information.

[0717] The processing flow will be explained below.

[0718] Reception duties during normal times

[0719] Step 1:

[0720] The user speaks into the smart speaker device about the reason for their visit.

[0721] For example: "I'm here to see the mayor."

[0722] Step 2:

[0723] The device accepts voice input and uses a speech recognition module to convert the user's speech into text data.

[0724] Speech to text: "I'm here to see the mayor."

[0725] Step 3:

[0726] The device analyzes the text data and determines the appropriate response based on the reason for the visit.

[0727] Analyze reason for visit: "Visiting the Mayor"

[0728] Step 4:

[0729] The terminal generates a text response based on the results of the analysis.

[0730] Response generation: "We will direct you to the appropriate department."

[0731] Step 5:

[0732] The terminal converts the generated text response into speech and plays it back.

[0733] Text-to-speech: "We will direct you to the appropriate department."

[0734] Audio conference during normal times

[0735] Step 1:

[0736] The user speaks to the terminal, "Please start the audio conference."

[0737] Step 2:

[0738] The terminal converts the voice into text data.

[0739] Voice-to-text: "Start your audio conference."

[0740] Step 3:

[0741] The terminal sends text data to the central server to instruct the start of the audio conference.

[0742] Step 4:

[0743] The server receives the instructions, sets up the audio conference, obtains the necessary connection information, and sends it to the terminal.

[0744] Acquiring and sending destination information

[0745] Step 5:

[0746] The terminal prompts the necessary locations to join the audio conference.

[0747] Step 6:

[0748] The users hold audio conferences at the designated terminals.

[0749] Disaster information dissemination during disasters

[0750] Step 1:

[0751] The server collects disaster information from television, radio, and social media.

[0752] Step 2:

[0753] The server analyzes the collected information using a generative AI module and eliminates false information.

[0754] Step 3:

[0755] The server generates reliable and accurate information and converts it into voice data.

[0756] Step 4:

[0757] The server transmits disaster information to each terminal and instructs it to make voice announcements.

[0758] Step 5:

[0759] The terminal processes the transmitted disaster information using a voice synthesis system and announces it to the user.

[0760] Example announcement: "An earthquake of magnitude 7 has occurred. Beware of tsunamis."

[0761] Announcement of local information

[0762] Step 1:

[0763] The device obtains its current location using the built-in GPS module.

[0764] Step 2:

[0765] The terminal transmits the acquired location information to a central server.

[0766] Location: "Your current location is XXX, YYY."

[0767] Step 3:

[0768] The server collects and analyzes information about surrounding disasters based on location information.

[0769] Step 4:

[0770] The server analyzes the collected peripheral information and selects and generates important information.

[0771] Step 5:

[0772] The server transmits the generated information to each terminal and instructs it to make a voice announcement.

[0773] Step 6:

[0774] The terminal synthesizes the received information into voice and announces it to the user.

[0775] Example announcement: "Heavy rain is causing flooding in this area. Please evacuate to higher ground."

[0776] Information sharing between bases

[0777] Step 1:

[0778] The user speaks to the terminal, "Please start an emergency meeting."

[0779] Step 2:

[0780] The terminal converts the voice into text data.

[0781] Step 3:

[0782] The terminal transmits text data to the central server, instructing the start of an emergency meeting.

[0783] Step 4:

[0784] The server sets up an emergency meeting, acquires connection destination information, and transmits it to the terminal.

[0785] Step 5:

[0786] The terminal prompts necessary locations to participate in the emergency meeting.

[0787] Step 6:

[0788] The user holds an emergency meeting at the designated terminal.

[0789] In this way, the smart speaker system of the present invention can provide quick and accurate information to local government facilities and local residents both during normal times and during disasters.

[0790] Example 1

[0791] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0792] The present invention aims to provide a system that eliminates false information and provides highly reliable information in situations where rapid and accurate information provision is required during disasters. It also aims to improve business efficiency and facilitate the implementation of voice conferences during normal times.

[0793] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0794] In this invention, the server includes means for accepting voice input from a user, means for converting the voice input into text data, means for analyzing the text data and generating information to be addressed, means for converting the generated information into voice and outputting it, means for communicating with other locations by voice, means for collecting information during a disaster, means for analyzing the collected information, eliminating false information, and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, means for generating and selecting the information using a generative AI model, and means including a voice recognition module and a voice synthesis module. This enables rapid and accurate information provision, improved business efficiency, and smooth voice conferences during disasters and normal times.

[0795] A "means for accepting voice input" is a device for detecting and recording voices uttered by a user.

[0796] The "means for converting voice input into text data" refers to software or algorithms for analyzing acquired voice data and converting it into text information.

[0797] "Means for analyzing text data and generating corresponding information" refers to a system that uses converted text data to understand and analyze the user's intentions and requests, and generates appropriate responses and guidance.

[0798] The "means for converting the generated information into voice and outputting it" refers to a voice synthesis technology and its output mechanism for reproducing text information as voice.

[0799] "Communication means for performing voice communication with other locations" refers to technology and systems for performing voice communication with multiple remote locations via a network.

[0800] "Means of collecting information during disasters" refers to systems and technologies for collecting relevant information from various sources when a disaster occurs.

[0801] "Means for analyzing collected information, eliminating false information, and selecting applicable information" refers to technology that evaluates the credibility of collected information, eliminates incorrect information, and selects highly reliable information.

[0802] "Means for obtaining current location information" refers to a system that uses GPS or other location information acquisition technology to determine the device's current location.

[0803] "Means for regularly updating disaster information in the surrounding area based on the current location" refers to a system that continuously obtains information on the disaster situation in the surrounding area based on the acquired location information and provides the latest information.

[0804] "Means for utilizing generative AI models" refers to systems that utilize artificial intelligence technology to generate appropriate information from collected and analyzed data.

[0805] A "voice recognition module" is a technology and software for analyzing voice data and converting it into text data.

[0806] A "voice synthesis module" is a technology and software for converting text data into voice data and playing it back.

[0807] The present invention relates to a system that receives, converts, and analyzes voice input, and converts the generated information into voice for output. Specific ways in which the present invention can be implemented are described below.

[0808] Hardware Configuration

[0809] 1. Smart speaker device:

[0810] A microphone to accept voice input

[0811] Speakers for playing audio

[0812] GPS module for obtaining location information

[0813] 2. Central Server:

[0814] High-performance server for data processing and analysis

[0815] A network interface for communicating with smart speaker devices in various locations

[0816] 3. Communications infrastructure:

[0817] A fast and stable internet connection

[0818] Software Configuration

[0819] 1. Speech Recognition Module:

[0820] Natural language processing (NLP) techniques for converting voice input into text data (e.g., popular cloud-based speech recognition services)

[0821] 2. Generative AI Model:

[0822] Artificial intelligence techniques (e.g., generative AI models) for analyzing text data and generating appropriate responses or information

[0823] 3. Speech synthesis module:

[0824] Speech synthesis technology to convert the generated text data into speech (e.g., a cloud-based speech synthesis service)

[0825] 4. Disaster information collection and analysis system:

[0826] Software that collects and analyzes information from various sources (e.g., television, radio, social media)

[0827] Operation overview

[0828] Reception duties during normal times

[0829] 1. Voice input and analysis:

[0830] The user speaks into the smart speaker device and explains the reason for their visit, for example, "I'm here to see the mayor."

[0831] The device accepts voice input and converts the voice into text data using a voice recognition module.

[0832] The converted text data is sent to a central server for analysis to understand the reason for the visit.

[0833] 2. Responses and Guidance:

[0834] The server generates a response to the user based on the analysis results.

[0835] The device uses a speech synthesis module to play back the response aloud.

[0836] Example: "We will refer you to the appropriate department."

[0837] Audio conference during normal times

[0838] 1. Start an audio conference:

[0839] A user says, "Start audio conference."

[0840] The device converts the speech into text and sends it to a central server.

[0841] The server sets up the audio conference and sends the necessary connection information to the terminal.

[0842] 2. Audio Conference Procedures:

[0843] An audio communication session is initiated between the terminals, and the users are audio conferencing.

[0844] Disaster information dissemination during disasters

[0845] 1. Information Collection and Analysis:

[0846] The server collects disaster information from various sources and uses a generative AI model for analysis to eliminate false information.

[0847] 2. Information Generation and Announcement:

[0848] The server generates reliable information and sends it to the terminal.

[0849] The terminal uses voice synthesis technology to announce information to the user.

[0850] Example: "An earthquake of magnitude 7 has occurred. Beware of tsunamis."

[0851] Announcement of local information

[0852] 1. Obtaining and sending location information:

[0853] The device uses a GPS module to obtain its current location information and transmits it to a central server.

[0854] 2. Peripheral information generation and announcement:

[0855] The server collects and analyzes information about disasters in the surrounding area based on location information, selects important information, and sends it to the terminal.

[0856] The device uses voice synthesis technology to periodically announce information to the user.

[0857] For example: "Heavy rain is causing flooding in this area. Please evacuate to higher ground."

[0858] Specific examples (related to generative AI models)

[0859] Prompt Sentence Examples

[0860] "When a user asks a smart speaker how to respond in the event of a disaster, the system generates prompts to provide reliable information."

[0861] Specific prompt sentences

[0862] "A magnitude 7 earthquake has occurred. Please provide evacuation routes and precautions to users living in areas where a tsunami may occur."

[0863] With the above-described configuration and operation, the smart speaker system of the present invention enables prompt and accurate information provision to local government facilities and local residents both during normal times and in times of disaster.

[0864] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0865] Step 1: Accepting voice input

[0866] The user speaks into the smart speaker device. The device uses a microphone to capture voice data. The input is the user's voice data, and the output is the captured voice data. This voice data is sent to the voice recognition module in the next step.

[0867] Step 2: Convert voice to text

[0868] The terminal sends the acquired voice data to the voice recognition module, which converts the voice data into text data. The input is the acquired voice data, and the output is text data. This text data is analyzed in the next step.

[0869] Step 3: Analyzing the text data

[0870] The terminal sends the converted text data to a central server, which then analyzes the text data using a natural language processing (NLP) module to understand the user's intent. For example, the text data "I've come to see the mayor" is analyzed to determine the appropriate department to respond to the request. The input is text data, and the output is the analysis result.

[0871] Step 4: Generate response text

[0872] The server generates a response to the user based on the analysis results. A generative AI model is used for this generation. The input is the analysis results, and the output is response text data. For example, the generated text might say, "We will direct you to the relevant department."

[0873] Step 5: Respond with text-to-speech

[0874] The terminal sends the response text data received from the server to the speech synthesis module. The speech synthesis module converts the text data into voice data and plays it back to the user through a speaker. The input is the response text data, and the output is voice data. For example, the voice may say, "We will direct you to the relevant department."

[0875] Step 6: Start an audio conference

[0876] The user speaks to the terminal, "Please start the audio conference." The terminal converts the speech into text data and sends it to the central server. The input is the user's voice data, and the output is the instruction text to start the audio conference.

[0877] Step 7: Configure the server for audio conferencing

[0878] The server receives the voice conference start instruction and sets up the voice conference. It obtains the necessary connection information using VOIP technology and sends it to the terminal. The input is the voice conference start instruction text, and the output is the connection information.

[0879] Step 8: Start a voice communication session

[0880] Each terminal starts a voice communication session based on the connection destination information from the server. This allows users to hold a voice conference. The input is the connection destination information, and the output is a voice communication session.

[0881] Step 9: Collect and analyze disaster information

[0882] The server collects disaster information from sources such as television, radio, and social media. It then analyzes the collected information using a generative AI model to eliminate false information. The input is disaster information data, and the output is highly reliable disaster information.

[0883] Step 10: Disaster information generation and announcement

[0884] The server generates highly reliable information after analysis and sends it to the terminal. The terminal converts the received information into voice data using voice synthesis technology and announces it to the user. The input is highly reliable disaster information, and the output is a voice announcement of the disaster information. For example, the announcement might say, "An earthquake with a seismic intensity of 7 has occurred. Beware of tsunamis."

[0885] Step 11: Obtaining and sending location information

[0886] The device acquires its current location information using the built-in GPS module and sends it to the server. The input is the location information, and the output is the data sent to the server.

[0887] Step 12: Generate and announce surrounding information

[0888] The server collects and analyzes information about surrounding disasters based on the received location information. It selects important information and sends it to the terminal, which then uses voice synthesis technology to announce it to the user. The input is location information, and the output is a voice announcement of surrounding disaster information. For example, the announcement might say, "Heavy rain is causing flood damage in this area. Please evacuate to higher ground."

[0889] This explains the process by which the system of the present invention provides rapid and accurate information while clarifying the specific operations at each processing step.

[0890] (Application example 1)

[0891] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0892] In modern society, there is a demand for fast and accurate information in the event of a disaster. However, with conventional methods, the collection, analysis, and distribution of disaster information are not centralized, which creates the risk of erroneous information spreading. Furthermore, it is difficult to provide individual disaster information based on real-time location information. This can lead to delays in evacuation behavior and the spread of anxiety-inducing information.

[0893] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0894] In this invention, the server includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating appropriate information, means for converting the generated information into voice and outputting it, means for communicating with other bases through voice, means for collecting information during a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, means for communicating with a central server to acquire and analyze disaster information, means for associating the acquired location information with the disaster information and notifying the user of the association by voice, a communication module for acquiring data from the server, and means for providing disaster information in real time using a voice assistant, thereby enabling accurate and prompt information provision during a disaster.

[0895] A "means for accepting voice input" is a device or software for recognizing and capturing voice instructions from a user.

[0896] "Means for converting voice input into text data" refers to technology for analyzing received voice data and converting it into text information.

[0897] The "means for analyzing text data and generating corresponding information" is a function for processing the converted text data and extracting and generating the necessary information.

[0898] The "means for converting the generated information into voice and outputting it" is a system for converting the generated information into synthesized voice and letting the user hear it.

[0899] "Communication means for performing voice communication with other locations" refers to a network function for exchanging voice data between multiple locations.

[0900] The "information gathering means in the event of a disaster" is a device or system for acquiring relevant information in the event of a disaster.

[0901] "Means for analyzing collected information and selecting applicable information" refers to the function of examining collected data and selecting useful information.

[0902] "Means for obtaining current location information" refers to technology for identifying the current location of a user or device.

[0903] "Means for regularly updating information about disasters in the surrounding area based on the current location" is a function that continuously reflects the latest information about disasters in the surrounding area based on the acquired location information.

[0904] "Means for communicating with a central server to obtain and analyze disaster information" refers to a system for obtaining disaster information from a central database and reviewing and processing that information.

[0905] "Means of associating acquired location information with disaster information and notifying the user as voice" refers to a function that compares location data with disaster information and notifies the user of the generated information as voice.

[0906] A "communication module for acquiring data from a server" is a device for acquiring information from a server via a network.

[0907] "Means of providing disaster information in real time using voice assistants" is a system that uses AI technology to provide disaster information instantly via voice.

[0908] This invention is a smart assistant system that accepts voice input, analyzes it, and outputs voice. Its purpose is to provide accurate information in real time, especially during disasters. The system works in conjunction with several main modules.

[0909] A user uses a smartphone to provide voice input. The smartphone uses a microphone to capture the speech and converts it into text data using speech recognition software (e.g., the speech_recognition library). The converted text data is then analyzed (e.g., using natural language processing techniques) to generate the requested information.

[0910] Next, the smartphone sends the acquired location information (e.g., using the geopy library to acquire GPS data) to the server, which then collects disaster information from a central database (e.g., using the requests library to acquire data from an API), analyzes the collected information using a generative AI model (e.g., a natural language generation model), and selects applicable information.

[0911] The server generates reliable information after analysis and sends it to the smartphone, which receives the data and announces the information to the user as voice using speech synthesis technology (e.g., the pyttsx3 library).

[0912] As a concrete example, suppose a user speaks to their smartphone, saying, "Please tell me the disaster information for my current location." In this case, the voice input is converted into text, and "disaster information" is recognized as the trigger word. The smartphone acquires the current location information and sends it to the server. The server collects and analyzes the latest disaster information around the current location, and generates accurate information such as, "An earthquake with a seismic intensity of 6 has occurred. Please evacuate immediately." The smartphone then announces this information to the user as voice.

[0913] Using generative AI models, it is possible to filter out misinformation and provide reliable information. An example prompt is as follows:

[0914] Please create the following Python code that can handle speech recognition and speech synthesis. The code will retrieve the latest disaster information related to the user's current location from the server and announce it by voice. The input speech is in Japanese.

[0915] In this way, the present invention makes it possible to provide accurate and prompt information in the event of a disaster, thereby ensuring the safety of users.

[0916] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0917] Step 1:

[0918] The user inputs voice commands using the smartphone's microphone, including commands such as "Please tell me disaster information for my current location."

[0919] Step 2:

[0920] The device accepts voice input and converts the voice into text data using voice recognition technology (e.g., the speech_recognition library). The input here is the user's voice data, and the output is a command in text format. Specifically, the device captures the user's voice from the microphone and converts it into text.

[0921] Step 3:

[0922] The terminal analyzes the converted text data and extracts the information that needs to be handled. For example, "disaster information" is recognized as a command word. The input is text data, and the output is a command word based on the analysis results. Specifically, the terminal performs keyword extraction and command analysis of the text.

[0923] Step 4:

[0924] The device uses a GPS module to obtain current location information. The input is GPS data, and the output is the current latitude and longitude information. Specifically, the device obtains the coordinates of the current location from the GPS sensor.

[0925] Step 5:

[0926] The device sends the acquired location information to a central server. The input is location data, and the output is communication packets such as HTTP requests. Specifically, the data is sent to the server via the Internet.

[0927] Step 6:

[0928] The server collects the latest disaster information based on the received location information. The collection source is television, radio, social media APIs, etc. The input is location information, and the output is the collected disaster information. Specifically, it makes an API call and retrieves the corresponding data.

[0929] Step 7:

[0930] The server analyzes the collected disaster information using a generative AI model, filtering out false information and selecting reliable information. The input is raw data, and the output is reliable information as the result of analysis. Specifically, the server performs data analysis and filtering using an AI model.

[0931] Step 8:

[0932] The server sends the analyzed information to the terminal. The input is the analysis result data, and the output is a communication packet. Specifically, the analyzed data is sent as an HTTP response.

[0933] Step 9:

[0934] The device receives the information from the server and announces it to the user as voice using speech synthesis technology (e.g., the pyttsx3 library). The input is the analysis result data, and the output is voice data. Specifically, the device converts text into voice and outputs it from the speaker.

[0935] Step 10:

[0936] The user receives voice information provided by the device and takes appropriate action in the event of a disaster. The input is voice data, and the output is the user's actions. Specifically, the user takes evacuation action by following the voice instructions from the device.

[0937] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0938] The present invention is a system that includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating information to be responded to, means for converting the generated information into voice and outputting it, means for communicating with other bases through voice, means for collecting information in the event of a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating information about disasters in the surrounding area based on the current location, and an emotion engine that recognizes the user's emotions. This system enables the rapid and accurate provision of information between local government facilities and to local residents both during normal times and during disasters.

[0939] System Configuration

[0940] The system includes the following main modules:

[0941] 1. Smart speaker device

[0942] 2. Central Server

[0943] 3. Multimodal AI Module

[0944] 4. Emotion Engine

[0945] Program processing

[0946] 1. Reception duties during normal times

[0947] Voice input and analysis

[0948] The user speaks the reason for their visit into the smart speaker device. The device accepts the voice input and converts the voice into text data using a speech recognition module. The converted text data is then analyzed to understand the reason for the visit.

[0949] emotion recognition

[0950] The emotion engine analyzes the user's voice data and recognizes the user's emotional state.

[0951] Response and guidance

[0952] Based on the analysis and emotion recognition results, the device generates a text response, adjusting the content and tone of the response depending on the results of the emotion engine.

[0953] For example, if the user is nervous, the response may be provided in a relaxing tone.

[0954] Example response: "We will direct you to the appropriate department."

[0955] Speech synthesis

[0956] The generated text response is converted into speech and played to the user in real time.

[0957] 2. Audio conferences during normal times

[0958] Starting an Audio Conference

[0959] The user speaks to the terminal, "Please start the audio conference." The voice is converted into text data, and the data instructing the central server to start the audio conference is sent.

[0960] Processing on the server

[0961] The server sets up an audio conference based on the instructions, obtains the necessary connection information, and sends it to the terminal. The terminal then prompts the necessary locations to join the audio conference.

[0962] Audio conference call

[0963] The terminals at the participating locations start an audio communication session, and the users conduct an audio conference.

[0964] 3. Disaster information dissemination during disasters

[0965] Collection of information

[0966] The server collects disaster information from television, radio, social media, etc. The collected information is analyzed using a generative AI module to eliminate false information.

[0967] Producing accurate information

[0968] After analysis, reliable information is generated and sent to the terminal as voice data.

[0969] Information voice announcement

[0970] The device processes the received disaster information using a voice synthesis system and announces it to the user.

[0971] For example, an announcement may be made such as, "An earthquake of magnitude 7 has occurred. Beware of tsunamis."

[0972] Announcement adjustment by emotion recognition

[0973] emotion recognition

[0974] The emotion engine recognizes the user's emotional state and assesses the user's stress level, especially in times of disaster.

[0975] Response Adjustment

[0976] Based on the results of the emotion engine, the content of the voice notification is adjusted to provide information in a tone and content that reduces the user's stress.

[0977] For example, if a user is in a panic, the announcement will be something like, "Please stay calm. We will provide you with the current information."

[0978] Announcement of local information

[0979] Collection and transmission of location information

[0980] The device uses its built-in GPS module to obtain its current location and transmit it to a central server.

[0981] Generate surrounding information

[0982] The server collects and analyzes information about the surrounding area based on location information, and selects and generates important information. The generated information is then sent to the device as voice data.

[0983] Regular updates and announcements

[0984] The terminal periodically announces updated information to the user using voice synthesis technology.

[0985] For example, an announcement may be made such as, "Heavy rain is causing flooding in this area. Please evacuate to higher ground."

[0986] Information sharing between bases

[0987] Emergency meeting begins

[0988] The user speaks to the terminal, "Please start an emergency meeting." The voice is converted into text data, and the data instructing the central server to start an emergency meeting is sent.

[0989] Processing on the server

[0990] The server sets up an emergency meeting, obtains the necessary connection information, and sends it to the terminal. The terminal then prompts the necessary locations to join the emergency meeting.

[0991] Emergency meeting proceedings

[0992] The terminals at the participating locations start a voice communication session, and the users hold an emergency conference.

[0993] As a result, the smart speaker system of the present invention can provide prompt and accurate information to local government facilities and local residents both during normal times and during disasters. Furthermore, the introduction of an emotion engine enables flexible responses according to the user's psychological state, helping to reduce panic, especially during disasters.

[0994] The processing flow will be explained below.

[0995] Reception duties during normal times

[0996] Step 1:

[0997] The user speaks into the smart speaker device about the reason for their visit.

[0998] For example: "I'm here to see the mayor."

[0999] Step 2:

[1000] The device accepts voice input and uses a speech recognition module to convert the user's speech into text data.

[1001] Speech to text: "I'm here to see the mayor."

[1002] Step 3:

[1003] The device analyzes the text data, determines the reason for the visit, and determines the appropriate response.

[1004] Analyze reason for visit: "Visiting the Mayor"

[1005] Step 4:

[1006] The device simultaneously sends the voice data to an emotion engine to recognize the user's emotions, for example, whether they are nervous.

[1007] Step 5:

[1008] The device generates a text response based on the analysis and emotion recognition results, adjusting the content and tone of the response depending on the results of the emotion engine.

[1009] Produced response: "I'll connect you to the right person." Plus, a relaxing tone

[1010] Step 6:

[1011] The terminal converts the generated text response into speech and plays it to the user in real time.

[1012] Audio conference during normal times

[1013] Step 1:

[1014] The user speaks to the terminal, "Please start the audio conference."

[1015] Step 2:

[1016] The terminal converts the voice into text data.

[1017] Voice-to-text: "Start your audio conference."

[1018] Step 3:

[1019] The terminal sends text data to the central server to instruct the start of the audio conference.

[1020] Step 4:

[1021] The server receives the instructions, sets up the audio conference, obtains the necessary connection information, and sends it to the terminal.

[1022] Acquiring and sending destination information

[1023] Step 5:

[1024] The terminal prompts the necessary locations to join the audio conference.

[1025] Step 6:

[1026] The terminal sends the user's voice to an emotion engine to recognize emotions during the meeting.

[1027] Step 7:

[1028] The server provides information to support the progress of the conference based on the results of the emotion engine, as needed.

[1029] Step 8:

[1030] The users hold audio conferences at the designated terminals.

[1031] Disaster information dissemination during disasters

[1032] Step 1:

[1033] The server collects disaster information from television, radio, and social media.

[1034] Step 2:

[1035] The server analyzes the collected information using a generative AI module and eliminates false information.

[1036] Step 3:

[1037] The server generates reliable and accurate information, converts it into voice data, and transmits it to the terminal.

[1038] Step 4:

[1039] The server transmits disaster information to each terminal and instructs it to make voice announcements.

[1040] Step 5:

[1041] The device processes the transmitted disaster information using a voice synthesis system and checks the user's emotional state using an emotion engine.

[1042] Step 6:

[1043] The terminal adjusts the announcement depending on the user's emotional state.

[1044] Example: "There has been a magnitude 7 earthquake. Beware of tsunamis." (Calm tone)

[1045] Step 7:

[1046] The user receives disaster information from the terminal.

[1047] Announcement of local information

[1048] Step 1:

[1049] The device obtains its current location using the built-in GPS module.

[1050] Step 2:

[1051] The terminal transmits the acquired location information to a central server.

[1052] Location: "Your current location is XXX, YYY."

[1053] Step 3:

[1054] The server collects and analyzes information about surrounding disasters based on location information.

[1055] Step 4:

[1056] The server analyzes the collected peripheral information and selects and generates important information.

[1057] Step 5:

[1058] The server transmits the generated information to each terminal and instructs it to make a voice announcement.

[1059] Step 6:

[1060] The terminal synthesizes the received information into voice and uses an emotion engine to recognize the user's emotional state.

[1061] Step 7:

[1062] The terminal adjusts the announcements according to the emotional state and provides information to the user.

[1063] For example: "Heavy rainfall is causing flooding in this area. Please seek refuge on higher ground." (Reassuring tone)

[1064] Information sharing between bases

[1065] Schedule an emergency meeting

[1066] Step 1:

[1067] The user speaks to the terminal, "Please start an emergency meeting."

[1068] Step 2:

[1069] The terminal converts the voice into text data.

[1070] Step 3:

[1071] The terminal transmits text data to the central server, instructing the start of an emergency meeting.

[1072] Step 4:

[1073] The server sets up an emergency meeting, obtains the necessary connection information, and transmits it to the terminal.

[1074] Step 5:

[1075] The terminal prompts necessary locations to participate in the emergency meeting.

[1076] Step 6:

[1077] The terminal sends the user's voice to an emotion engine to recognize emotions during an emergency meeting.

[1078] Step 7:

[1079] The server provides information to assist in the progress of the emergency meeting, if necessary, based on the results of the emotion engine.

[1080] Step 8:

[1081] The user holds an emergency meeting at the designated terminal.

[1082] In this way, the smart speaker system of the present invention can provide prompt and accurate information to local government facilities and local residents both during normal times and during disasters.Furthermore, the introduction of an emotion engine makes it possible to respond flexibly to the user's emotional state, which is a distinctive feature of the system, reducing the user's psychological burden.

[1083] Example 2

[1084] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1085] While systems using conventional voice recognition technology are required to provide accurate and prompt emergency information during disasters, in reality, they face challenges in effectively communicating information due to the inability to properly assess the user's stress level and the inclusion of false information. Furthermore, in normal reception work and voice conferences, it is difficult to respond according to the user's emotional state, leading to a demand for improved user experience.

[1086] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating information to be handled, means for converting the generated information into voice and outputting it, communication means for performing voice communication with other bases, means for collecting information in the event of a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, and means for recognizing the user's emotional state and adjusting the response content. This makes it possible to quickly provide accurate and appropriate information in the event of a disaster, and to flexibly respond to the user's emotional state in normal times.

[1087] The "means for accepting voice input" is a device that has the function of capturing the voice spoken by the user as a digital signal.

[1088] "Means for converting voice input into text data" refers to technology for analyzing voice signals and converting them into corresponding text data.

[1089] "Means for analyzing text data and generating corresponding information" refers to technology that processes acquired text data and extracts and generates information based on the user's requests and intentions.

[1090] The "means for converting the generated information into voice and outputting it" is a device that has the function of converting text data into a voice signal using voice synthesis technology and playing it aloud to the user.

[1091] "Communication means for performing voice communication with other locations" refers to communication technology for transmitting and receiving voice data between multiple terminals and servers.

[1092] "Means of collecting information during disasters" refers to technology that has the function of collecting disaster-related information from various sources such as television, radio, and social media.

[1093] "Means for analyzing collected information and selecting applicable information" refers to the technology of processing collected information using data analysis technology and selecting highly reliable information.

[1094] "Means for obtaining current location information" refers to technology for identifying the current location using a GPS module or the like.

[1095] "Means for periodically updating surrounding disaster information based on the current location" refers to technology that continuously collects disaster information for the surrounding area based on the acquired location information and updates it to the latest information.

[1096] "Means for recognizing the user's emotional state and adjusting the response content" refers to technology that analyzes emotions from the user's voice and adjusts the response content and tone based on the results.

[1097] MODE FOR CARRYING OUT THE INVENTION

[1098] The present invention includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating corresponding information, means for converting the generated information into voice and outputting it, communication means for performing voice communication with other bases, means for collecting information in the event of a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, and means for recognizing the user's emotional state and adjusting the response content.

[1099] The system of the present invention includes the following main modules:

[1100] 1. A means of accepting voice input (smart speaker device)

[1101] Smart speaker devices have built-in microphones that accept voice input from users. For example, if you say, "I want to go to the government office," the voice is captured by the device as a digital signal.

[1102] 2. A means of converting voice input into text data

[1103] The device converts the received voice input into text using a speech recognition module such as Google Cloud Speech-to-Text, which analyzes the voice signal and converts it into text.

[1104] 3. A method for analyzing text data and generating relevant information

[1105] The converted text data is analyzed using natural language processing (NLP) technology to extract information based on the user's intent and requests. For example, if the user enters "I want to go to the government office," the system identifies "visiting the government office" as the reason for the visit.

[1106] 4. A means of converting the generated information into audio and outputting it

[1107] The text response generated based on the analysis results is converted into speech using a speech synthesis system such as Amazon Polly. The device then plays this speech back to the user, responding, for example, in a relaxing tone, saying, "We will direct you to the appropriate department."

[1108] 5. Communication means for voice communication with other locations

[1109] To enable voice communication between the device and other locations, a connection is set up using the Zoom API for each location, allowing users to hold voice conferences and communicate through the device.

[1110] 6. Information gathering methods during disasters

[1111] The server collects disaster information from sources such as television, radio, and social media. The collected information is then analyzed using OpenAI's generative AI model to filter out unreliable information. This process ensures accurate and prompt information provision.

[1112] 7. How we analyze the collected information and select applicable information

[1113] The collected disaster information is processed using data analysis technology to extract applicable and reliable information, which is then generated in text format and transmitted to the device as voice data.

[1114] 8. How to get current location information

[1115] The device uses its built-in GPS module to determine its current location and sends it to the server using the Google Maps API.

[1116] 9. A way to periodically update information about nearby disasters based on your current location

[1117] The server collects and analyzes disaster information from the surrounding area based on the acquired location information, and selects important information. This information is updated periodically and sent to the device as voice data.

[1118] 10. A way to recognize the user's emotional state and tailor responses

[1119] The emotion engine analyzes the user's voice data and recognizes the user's emotional state using tools such as IBM Watson Tone Analyzer. Particularly in times of disaster, the system assesses the user's stress level and panic and adjusts the content and tone of the response accordingly. For example, the response may be something like, "Please stay calm. We'll provide you with the latest information."

[1120] Examples of concrete examples and prompts

[1121] For example, if a user says to a smart speaker, "I want to go to the government office," the device converts the voice into text data and analyzes the reason for the visit. If the emotion engine determines that the user is nervous, it responds in a relaxing tone, saying, "I will guide you to the relevant department," and plays the response aloud.

[1122] An example prompt is:

[1123] The system converts what the user says into text and analyzes that text to understand the reason for the visit. It then analyzes the user's emotional state and responds by voice, adjusting the tone of the response as needed. For example, if the user says, "I'd like to go to the government office," the system responds in a relaxing tone, saying, "I'll take you to the appropriate department."

[1124] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1125] Processing Steps

[1126] Step 1: Accepting voice input

[1127] Subject: Terminal

[1128] Specific behavior:

[1129] The user speaks into the smart speaker device and says, "I would like to go to the city hall."

[1130] The device captures audio using its built-in microphone.

[1131] input:

[1132] User Voice

[1133] Data processing:

[1134] Converts audio signals into digital data.

[1135] output:

[1136] Digital audio data

[1137] Step 2: Convert audio data to text

[1138] Subject: Terminal

[1139] Specific behavior:

[1140] The device converts the voice data into text data using the Google Cloud Speech-to-Text module.

[1141] input:

[1142] Digital audio data

[1143] Data processing:

[1144] The voice signal is analyzed and converted into corresponding text data.

[1145] output:

[1146] Converted text data (e.g., "I would like to go to the government office.")

[1147] Step 3: Analyzing the text data

[1148] Subject: Terminal

[1149] Specific behavior:

[1150] The device uses natural language processing (NLP) technology to analyze the text data and understand the reason for the visit.

[1151] input:

[1152] Converted text data

[1153] Data processing:

[1154] Through text data analysis, the reason for the visit is identified (e.g., "visiting government office").

[1155] output:

[1156] Analyzed information (e.g., "visit to government office")

[1157] Step 4: Emotion Recognition

[1158] Subject: Terminal

[1159] Specific behavior:

[1160] The emotion engine uses IBM Watson Tone Analyzer to analyze the user's voice data and recognize their emotional state.

[1161] input:

[1162] Converted text data

[1163] Captured audio data

[1164] Data processing:

[1165] Perform tone and sentiment analysis on voice and text data.

[1166] output:

[1167] Emotional state analysis results (e.g., tension)

[1168] Step 5: Response generation and adjustment

[1169] Subject: Terminal

[1170] Specific behavior:

[1171] Based on the analysis and emotion recognition results, the device generates a text response.

[1172] Adjust the content and tone of your response depending on the results of the emotion engine.

[1173] input:

[1174] Analyzed information (e.g., "visit to government office")

[1175] Emotional state analysis results (e.g., tension)

[1176] Data processing:

[1177] Generate text responses and adjust tone

[1178] output:

[1179] Tailored text responses (e.g., "We'll direct you to the appropriate department")

[1180] Step 6: Speech synthesis and response playback

[1181] Subject: Terminal

[1182] Specific behavior:

[1183] The device converts the generated text response into speech using a speech synthesis system such as Amazon Polly.

[1184] Plays a sound to the user.

[1185] input:

[1186] Tailored text responses (e.g., "We'll direct you to the appropriate department")

[1187] Data processing:

[1188] Convert text data into audio signals.

[1189] output:

[1190] Voice response (e.g., "We will direct you to the appropriate department")

[1191] Step 7: Collect disaster information

[1192] Subject: Server

[1193] Specific behavior:

[1194] The server collects disaster information from television, radio, social media, etc.

[1195] input:

[1196] Disaster-related information data from various sources

[1197] Data processing:

[1198] Disaster information collection and data conversion

[1199] output:

[1200] Collected disaster information data

[1201] Step 8: Analysis and selection of disaster information

[1202] Subject: Server

[1203] Specific behavior:

[1204] The server uses OpenAI's generative AI model to analyze the collected disaster information and eliminate false information.

[1205] Select reliable disaster information.

[1206] input:

[1207] Collected disaster information data

[1208] Data processing:

[1209] Data analysis and information selection

[1210] output:

[1211] Reliable disaster information

[1212] Step 9: Obtaining and sending location information

[1213] Subject: Terminal

[1214] Specific behavior:

[1215] The device uses the built-in GPS module to obtain its current location.

[1216] The acquired location information is sent to the server.

[1217] input:

[1218] Location data from the GPS module

[1219] Data processing:

[1220] Acquiring location information and improving data formats

[1221] output:

[1222] Current location information

[1223] Step 10: Collect and generate information on surrounding disasters

[1224] Subject: Server

[1225] Specific behavior:

[1226] The server collects and analyzes information about surrounding disasters based on location information.

[1227] input:

[1228] Current location information

[1229] Data processing:

[1230] Collection and analysis of information on nearby disasters

[1231] output:

[1232] Generated disaster information data

[1233] Step 11: Disaster Information Updates and Announcements

[1234] Subject: Terminal

[1235] Specific behavior:

[1236] The disaster information received by the device is converted into speech using a speech synthesis system such as Amazon Polly.

[1237] Users will be notified of the latest disaster information on a regular basis.

[1238] input:

[1239] Generated disaster information data

[1240] Data processing:

[1241] Audio data generation and announcement of disaster information

[1242] output:

[1243] Voice announcements (e.g., "Heavy rain is causing flooding in this area. Please evacuate to higher ground.")

[1244] Step 12: Recognize the user's emotional state and adjust your response

[1245] Subject: Terminal

[1246] Specific behavior:

[1247] The emotion engine analyzes the user's voice data and assesses their emotional state.

[1248] Tailor your response and tone based on the user's stress level or panic.

[1249] input:

[1250] User voice data

[1251] Emotional state analysis results

[1252] Data processing:

[1253] Emotional state assessment and response adjustment

[1254] output:

[1255] Tailored voice responses (e.g., "Please remain calm. We'll provide you with the latest information.")

[1256] As a result, the present invention enables accurate and prompt provision of information in the event of a disaster, and realizes flexible responses according to the user's emotional state during normal times.

[1257] (Application example 2)

[1258] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1259] Conventional disaster information systems have problems with insufficient real-time information provision and inability to provide information that takes into account the user's emotional state. This makes it difficult for users to take appropriate action in an emergency, increasing the likelihood of them falling into a state of panic. Furthermore, the provision of individual disaster information based on location information is insufficient, making it impossible to encourage appropriate responses based on information specific to the area. It is desirable to solve these problems and ensure the safety of users.

[1260] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating information to be addressed, means for converting the generated information into voice and outputting it, means for communicating with other locations via voice, means for collecting information during a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, emotion analysis means for recognizing emotions, and means for adjusting the content and tone of a voice response based on the emotional state recognized by the emotion analysis means. This enables accurate information to be provided in real time during disasters and emergencies, enabling flexible responses according to the user's emotional state. Furthermore, based on individual location information, disaster information specific to the region can be provided, encouraging appropriate action.

[1261] "Means for accepting voice input" refers to a device or software that has the function of detecting voice and capturing it as a digital signal.

[1262] A "means for converting speech input into text data" is a system or algorithm that analyzes received speech signals and converts them into corresponding text information.

[1263] The "means for analyzing text data and generating corresponding information" refers to a process or module that analyzes input character information and generates information to be provided to the user based on the content of that information.

[1264] The "means for converting generated information into audio and outputting it" is a technology for converting information generated on a text basis into audio and providing it to the user as audio.

[1265] "Communication means for performing voice communication with other locations" refers to communication technology and equipment for transmitting and receiving voice data between devices and systems located in different locations.

[1266] "Means of collecting information during disasters" refers to mechanisms and means for collecting useful information from outside when a disaster occurs.

[1267] "Means for analyzing collected information and selecting applicable information" refers to a system or algorithm for analyzing the various collected information and selecting information that is reliable and useful to the user.

[1268] "Means for obtaining current location information" refers to technology for measuring and obtaining the current location of the device (e.g., GPS module).

[1269] "Means for periodically updating surrounding disaster information based on current location" is a mechanism that uses acquired location information to periodically acquire and update surrounding disaster information.

[1270] "Emotion analysis means for recognizing emotions" refers to technologies and algorithms that analyze a user's voice or text and evaluate and recognize their emotional state.

[1271] The "means for adjusting the content and tone of the voice response based on the emotional state recognized by the emotion analysis means" is a function for appropriately adjusting the content and tone of the generated voice response according to the recognized emotional state of the user.

[1272] The present invention is a system for providing information during disasters and emergencies, and is composed of the following components: First, a smartphone, smart speaker, or other voice-enabled device is used as a means for users to input information by voice, allowing users to request information in real time.

[1273] To convert voice input into text data, we use the Google Speech-to-Text API. This API quickly converts voice data into text data. The text data is then analyzed and the corresponding information is generated. The software used here is the NLTK library, a Python natural language processing library. This library is used to analyze the user's input and generate the corresponding information.

[1274] The generated text data is converted into audio and provided to the user. Using the Google Text-to-Speech API, the generated text data is converted into high-quality audio. The audio output is provided through the speaker of a smartphone or smart speaker.

[1275] This system also has a communication means for voice communication with other locations. For example, when a user starts an audio conference, they can connect to other locations through a calling service such as Google Meet or Zoom and conduct voice communication.

[1276] In the event of a disaster, information is collected from television, radio, social media, etc., and analyzed to select applicable information. A generative AI model is used to analyze the collected information. This model eliminates false information and provides reliable information.

[1277] To obtain current location information, the built-in GPS module of the smartphone is used. Based on the obtained location information, information on disasters in the surrounding area is periodically updated and provided to the user. For example, heavy rain warnings and earthquake information for the area can be collected from the location information and notified to the user.

[1278] The IBM Watson Tone Analyzer is used to analyze emotions. This analyzes the user's voice and text data to recognize their emotional state. Based on the recognized emotional state, the content and tone of the voice response are adjusted. For example, if the user is nervous, information is provided in a slower, calmer tone.

[1279] Specific examples

[1280] A specific example is shown below. When a user says, "Please tell me the latest disaster information for this area," this speech is accepted as voice input and converted into text data using the Google Speech-to-Text API. The text data is then analyzed using the NLTK library, and the relevant disaster information is generated. The generated information is then converted into audio using the Google Text-to-Speech API and provided to the user in real time.

[1281] Example prompt sentence:

[1282] User says: "What are the latest disaster updates for this area?"

[1283] App response: Obtains location information, analyzes the user's emotional state with an emotion engine, and outputs collected disaster information in voice with adjusted tone.

[1284] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1285] Step 1:

[1286] The user inputs information by voice. The user speaks to a smartphone or smart speaker, saying something like, "Please tell me the latest disaster information for this area." The input data is voice data.

[1287] Step 2:

[1288] The device accepts voice input, records the audio as a digital signal, and then uses the Google Speech-to-Text API to convert this audio data into text data. The input is audio data, and the output is audio as text data.

[1289] Step 3:

[1290] The server receives the text data and analyzes it using Python's NLTK library. This analysis extracts important keywords and phrases from the input text. The data processing performed here involves analyzing the text and extracting the necessary information. The output is the analyzed text data.

[1291] Step 4:

[1292] The server generates appropriate disaster information based on the extracted keywords and phrases. Disaster information is collected using, for example, a disaster prevention information API. The generated information is expressed in text format. The input is analyzed text data, and the output is disaster information text data.

[1293] Step 5:

[1294] The server uses a generative AI model to analyze the collected disaster information and select applicable information. The generative AI model evaluates the reliability of the collected information and filters out unnecessary information. The input is text data of disaster information, and the output is reliable disaster information.

[1295] Step 6:

[1296] The server converts text data into audio data using the Google Text-to-Speech API to output the selected disaster information as audio. The input is reliable disaster information text data, and the output is audio data.

[1297] Step 7:

[1298] The voice data is provided to the user through the speaker of a smartphone or smart speaker. Conveniently, the generated voice is delivered to the user in real time. The output voice is then listened to by the user.

[1299] Step 8:

[1300] As a means of emotion analysis, IBM Watson Tone Analyzer is used to analyze the user's voice or text data to recognize their emotional state. The input is the user's voice or text data, and the output is evaluation data of the user's emotional state.

[1301] Step 9:

[1302] The server adjusts the content and tone of the voice response based on the emotion analysis data. The speed and tone of the voice change depending on the user's emotional state, reducing stress and panic in the user. The input is emotion analysis data, and the output is adjusted voice data.

[1303] For example:

[1304] When a user voice-inputs "I want disaster information," the speech is converted into text data by the Google Speech-to-Text API and analyzed by the Python NLTK library. The latest disaster information obtained from the disaster prevention information API is then converted into audio data by the Google Text-to-Speech API and provided to the user through the smartphone speaker. At the same time, the IBM Watson Tone Analyzer evaluates the user's emotional state and provides information in an appropriate tone.

[1305] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1306] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1307] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1308] [Third embodiment]

[1309] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1310] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1311] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1312] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1313] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1314] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1315] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1316] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1317] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1318] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1319] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1320] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1321] The present invention is a system including means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating information to be responded to, means for converting the generated information into voice and outputting it, means for communicating with other bases through voice, means for collecting information in the event of a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, and means for periodically updating surrounding disaster information based on the current location. This system enables the rapid and accurate provision of information between local government facilities and to local residents both during normal times and during disasters.

[1322] System Configuration

[1323] The system includes the following main modules:

[1324] 1. Smart speaker device

[1325] 2. Central Server

[1326] 3. Multimodal AI Module

[1327] Program processing

[1328] 1. Reception duties during normal times

[1329] Voice input and analysis

[1330] The user speaks the reason for their visit into the smart speaker device. The device accepts the voice input and converts the voice into text data using a speech recognition module. The converted text data is then analyzed to understand the reason for the visit.

[1331] For example, if a user says, "I'm here to see the mayor," the device will parse this text and generate directions to the appropriate department.

[1332] Response and guidance

[1333] Based on the analysis results, the terminal generates a text response and plays it back to the user in real time using speech synthesis technology.

[1334] As an example of a response, a voice saying "We will direct you to the relevant department" is played.

[1335] 2. Audio conferences during normal times

[1336] Starting an Audio Conference

[1337] The user speaks to the terminal, "Please start the audio conference." The terminal converts the speech into text and sends data to the central server instructing it to start the audio conference.

[1338] Processing on the server

[1339] The server sets up an audio conference based on the received instructions, obtains the necessary connection information, and sends it to the terminal. The terminal then prompts the necessary locations to join the audio conference.

[1340] Audio conference call

[1341] The terminals at the participating locations start an audio communication session, and the users conduct an audio conference.

[1342] 3. Disaster information dissemination during disasters

[1343] Collection of information

[1344] The server collects disaster information from television, radio, social media, etc. The collected information is analyzed using a generative AI module, and false information is filtered out.

[1345] Producing accurate information

[1346] After analysis, reliable information is generated and sent to the terminal as voice data.

[1347] Information voice announcement

[1348] The terminal announces the received disaster information to the user using voice synthesis technology.

[1349] For example, an announcement may be made such as, "An earthquake of magnitude 7 has occurred. Beware of tsunamis."

[1350] 4. Local area information announcements

[1351] Collection and transmission of location information

[1352] The device uses the built-in GPS module to obtain current location information, which is then sent to the server.

[1353] Generate surrounding information

[1354] The server collects and analyzes information about the surrounding area based on location information, and selects and generates important information, which is then sent to the device as voice data.

[1355] Regular updates and announcements

[1356] The terminal periodically announces updated information to the user using voice synthesis technology.

[1357] For example, an announcement may be made saying, "Heavy rain is causing flooding in this area. Please evacuate to higher ground."

[1358] As a result, the smart speaker system of the present invention will be able to provide prompt and accurate information to local government facilities and local residents both during normal times and during disasters. Furthermore, by using a generative AI model, the system is able to eliminate false information and provide highly reliable information.

[1359] The processing flow will be explained below.

[1360] Reception duties during normal times

[1361] Step 1:

[1362] The user speaks into the smart speaker device about the reason for their visit.

[1363] For example: "I'm here to see the mayor."

[1364] Step 2:

[1365] The device accepts voice input and uses a speech recognition module to convert the user's speech into text data.

[1366] Speech to text: "I'm here to see the mayor."

[1367] Step 3:

[1368] The device analyzes the text data and determines the appropriate response based on the reason for the visit.

[1369] Analyze reason for visit: "Visiting the Mayor"

[1370] Step 4:

[1371] The terminal generates a text response based on the results of the analysis.

[1372] Response generation: "We will direct you to the appropriate department."

[1373] Step 5:

[1374] The terminal converts the generated text response into speech and plays it back.

[1375] Text-to-speech: "We will direct you to the appropriate department."

[1376] Audio conference during normal times

[1377] Step 1:

[1378] The user speaks to the terminal, "Please start the audio conference."

[1379] Step 2:

[1380] The terminal converts the voice into text data.

[1381] Voice-to-text: "Start your audio conference."

[1382] Step 3:

[1383] The terminal sends text data to the central server to instruct the start of the audio conference.

[1384] Step 4:

[1385] The server receives the instructions, sets up the audio conference, obtains the necessary connection information, and sends it to the terminal.

[1386] Acquiring and sending destination information

[1387] Step 5:

[1388] The terminal prompts the necessary locations to join the audio conference.

[1389] Step 6:

[1390] The users hold audio conferences at the designated terminals.

[1391] Disaster information dissemination during disasters

[1392] Step 1:

[1393] The server collects disaster information from television, radio, and social media.

[1394] Step 2:

[1395] The server analyzes the collected information using a generative AI module and eliminates false information.

[1396] Step 3:

[1397] The server generates reliable and accurate information and converts it into voice data.

[1398] Step 4:

[1399] The server transmits disaster information to each terminal and instructs it to make voice announcements.

[1400] Step 5:

[1401] The terminal processes the transmitted disaster information using a voice synthesis system and announces it to the user.

[1402] Example announcement: "An earthquake of magnitude 7 has occurred. Beware of tsunamis."

[1403] Announcement of local information

[1404] Step 1:

[1405] The device obtains its current location using the built-in GPS module.

[1406] Step 2:

[1407] The terminal transmits the acquired location information to a central server.

[1408] Location: "Your current location is XXX, YYY."

[1409] Step 3:

[1410] The server collects and analyzes information about surrounding disasters based on location information.

[1411] Step 4:

[1412] The server analyzes the collected peripheral information and selects and generates important information.

[1413] Step 5:

[1414] The server transmits the generated information to each terminal and instructs it to make a voice announcement.

[1415] Step 6:

[1416] The terminal synthesizes the received information into voice and announces it to the user.

[1417] Example announcement: "Heavy rain is causing flooding in this area. Please evacuate to higher ground."

[1418] Information sharing between bases

[1419] Step 1:

[1420] The user speaks to the terminal, "Please start an emergency meeting."

[1421] Step 2:

[1422] The terminal converts the voice into text data.

[1423] Step 3:

[1424] The terminal transmits text data to the central server, instructing the start of an emergency meeting.

[1425] Step 4:

[1426] The server sets up an emergency meeting, acquires connection destination information, and transmits it to the terminal.

[1427] Step 5:

[1428] The terminal prompts necessary locations to participate in the emergency meeting.

[1429] Step 6:

[1430] The user holds an emergency meeting at the designated terminal.

[1431] In this way, the smart speaker system of the present invention can provide quick and accurate information to local government facilities and local residents both during normal times and during disasters.

[1432] Example 1

[1433] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1434] The present invention aims to provide a system that eliminates false information and provides highly reliable information in situations where rapid and accurate information provision is required during disasters. It also aims to improve business efficiency and facilitate the implementation of voice conferences during normal times.

[1435] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1436] In this invention, the server includes means for accepting voice input from a user, means for converting the voice input into text data, means for analyzing the text data and generating information to be addressed, means for converting the generated information into voice and outputting it, means for communicating with other locations by voice, means for collecting information during a disaster, means for analyzing the collected information, eliminating false information, and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, means for generating and selecting the information using a generative AI model, and means including a voice recognition module and a voice synthesis module. This enables rapid and accurate information provision, improved business efficiency, and smooth voice conferences during disasters and normal times.

[1437] A "means for accepting voice input" is a device for detecting and recording voices uttered by a user.

[1438] The "means for converting voice input into text data" refers to software or algorithms for analyzing acquired voice data and converting it into text information.

[1439] "Means for analyzing text data and generating corresponding information" refers to a system that uses converted text data to understand and analyze the user's intentions and requests, and generates appropriate responses and guidance.

[1440] The "means for converting the generated information into voice and outputting it" refers to a voice synthesis technology and its output mechanism for reproducing text information as voice.

[1441] "Communication means for performing voice communication with other locations" refers to technology and systems for performing voice communication with multiple remote locations via a network.

[1442] "Means of collecting information during disasters" refers to systems and technologies for collecting relevant information from various sources when a disaster occurs.

[1443] "Means for analyzing collected information, eliminating false information, and selecting applicable information" refers to technology that evaluates the credibility of collected information, eliminates incorrect information, and selects highly reliable information.

[1444] "Means for obtaining current location information" refers to a system that uses GPS or other location information acquisition technology to determine the device's current location.

[1445] "Means for regularly updating disaster information in the surrounding area based on the current location" refers to a system that continuously obtains information on the disaster situation in the surrounding area based on the acquired location information and provides the latest information.

[1446] "Means for utilizing generative AI models" refers to systems that utilize artificial intelligence technology to generate appropriate information from collected and analyzed data.

[1447] A "voice recognition module" is a technology and software for analyzing voice data and converting it into text data.

[1448] A "voice synthesis module" is a technology and software for converting text data into voice data and playing it back.

[1449] The present invention relates to a system that receives, converts, and analyzes voice input, and converts the generated information into voice for output. Specific ways in which the present invention can be implemented are described below.

[1450] Hardware Configuration

[1451] 1. Smart speaker device:

[1452] A microphone to accept voice input

[1453] Speakers for playing audio

[1454] GPS module for obtaining location information

[1455] 2. Central Server:

[1456] High-performance server for data processing and analysis

[1457] A network interface for communicating with smart speaker devices in various locations

[1458] 3. Communications infrastructure:

[1459] A fast and stable internet connection

[1460] Software Configuration

[1461] 1. Speech Recognition Module:

[1462] Natural language processing (NLP) techniques for converting voice input into text data (e.g., popular cloud-based speech recognition services)

[1463] 2. Generative AI Model:

[1464] Artificial intelligence techniques (e.g., generative AI models) for analyzing text data and generating appropriate responses or information

[1465] 3. Speech synthesis module:

[1466] Speech synthesis technology to convert the generated text data into speech (e.g., a cloud-based speech synthesis service)

[1467] 4. Disaster information collection and analysis system:

[1468] Software that collects and analyzes information from various sources (e.g., television, radio, social media)

[1469] Operation overview

[1470] Reception duties during normal times

[1471] 1. Voice input and analysis:

[1472] The user speaks into the smart speaker device and explains the reason for their visit, for example, "I'm here to see the mayor."

[1473] The device accepts voice input and converts the voice into text data using a voice recognition module.

[1474] The converted text data is sent to a central server for analysis to understand the reason for the visit.

[1475] 2. Responses and Guidance:

[1476] The server generates a response to the user based on the analysis results.

[1477] The device uses a speech synthesis module to play back the response aloud.

[1478] Example: "We will refer you to the appropriate department."

[1479] Audio conference during normal times

[1480] 1. Start an audio conference:

[1481] A user says, "Start audio conference."

[1482] The device converts the speech into text and sends it to a central server.

[1483] The server sets up the audio conference and sends the necessary connection information to the terminal.

[1484] 2. Audio Conference Procedures:

[1485] An audio communication session is initiated between the terminals, and the users are audio conferencing.

[1486] Disaster information dissemination during disasters

[1487] 1. Information Collection and Analysis:

[1488] The server collects disaster information from various sources and uses a generative AI model for analysis to eliminate false information.

[1489] 2. Information Generation and Announcement:

[1490] The server generates reliable information and sends it to the terminal.

[1491] The terminal uses voice synthesis technology to announce information to the user.

[1492] Example: "An earthquake of magnitude 7 has occurred. Beware of tsunamis."

[1493] Announcement of local information

[1494] 1. Obtaining and sending location information:

[1495] The device uses a GPS module to obtain its current location information and transmits it to a central server.

[1496] 2. Peripheral information generation and announcement:

[1497] The server collects and analyzes information about disasters in the surrounding area based on location information, selects important information, and sends it to the terminal.

[1498] The device uses voice synthesis technology to periodically announce information to the user.

[1499] For example: "Heavy rain is causing flooding in this area. Please evacuate to higher ground."

[1500] Specific examples (related to generative AI models)

[1501] Prompt Sentence Examples

[1502] "When a user asks a smart speaker how to respond in the event of a disaster, the system generates prompts to provide reliable information."

[1503] Specific prompt sentences

[1504] "A magnitude 7 earthquake has occurred. Please provide evacuation routes and precautions to users living in areas where a tsunami may occur."

[1505] With the above-described configuration and operation, the smart speaker system of the present invention enables prompt and accurate information provision to local government facilities and local residents both during normal times and in times of disaster.

[1506] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1507] Step 1: Accepting voice input

[1508] The user speaks into the smart speaker device. The device uses a microphone to capture voice data. The input is the user's voice data, and the output is the captured voice data. This voice data is sent to the voice recognition module in the next step.

[1509] Step 2: Convert voice to text

[1510] The terminal sends the acquired voice data to the voice recognition module, which converts the voice data into text data. The input is the acquired voice data, and the output is text data. This text data is analyzed in the next step.

[1511] Step 3: Analyzing the text data

[1512] The terminal sends the converted text data to a central server, which then analyzes the text data using a natural language processing (NLP) module to understand the user's intent. For example, the text data "I've come to see the mayor" is analyzed to determine the appropriate department to respond to the request. The input is text data, and the output is the analysis result.

[1513] Step 4: Generate response text

[1514] The server generates a response to the user based on the analysis results. A generative AI model is used for this generation. The input is the analysis results, and the output is response text data. For example, the generated text might say, "We will direct you to the relevant department."

[1515] Step 5: Respond with text-to-speech

[1516] The terminal sends the response text data received from the server to the speech synthesis module. The speech synthesis module converts the text data into voice data and plays it back to the user through a speaker. The input is the response text data, and the output is voice data. For example, the voice may say, "We will direct you to the relevant department."

[1517] Step 6: Start an audio conference

[1518] The user speaks to the terminal, "Please start the audio conference." The terminal converts the speech into text data and sends it to the central server. The input is the user's voice data, and the output is the instruction text to start the audio conference.

[1519] Step 7: Configure the server for audio conferencing

[1520] The server receives the voice conference start instruction and sets up the voice conference. It obtains the necessary connection information using VOIP technology and sends it to the terminal. The input is the voice conference start instruction text, and the output is the connection information.

[1521] Step 8: Start a voice communication session

[1522] Each terminal starts a voice communication session based on the connection destination information from the server. This allows users to hold a voice conference. The input is the connection destination information, and the output is a voice communication session.

[1523] Step 9: Collect and analyze disaster information

[1524] The server collects disaster information from sources such as television, radio, and social media. It then analyzes the collected information using a generative AI model to eliminate false information. The input is disaster information data, and the output is highly reliable disaster information.

[1525] Step 10: Disaster information generation and announcement

[1526] The server generates highly reliable information after analysis and sends it to the terminal. The terminal converts the received information into voice data using voice synthesis technology and announces it to the user. The input is highly reliable disaster information, and the output is a voice announcement of the disaster information. For example, the announcement might say, "An earthquake with a seismic intensity of 7 has occurred. Beware of tsunamis."

[1527] Step 11: Obtaining and sending location information

[1528] The device acquires its current location information using the built-in GPS module and sends it to the server. The input is the location information, and the output is the data sent to the server.

[1529] Step 12: Generate and announce surrounding information

[1530] The server collects and analyzes information about surrounding disasters based on the received location information. It selects important information and sends it to the terminal, which then uses voice synthesis technology to announce it to the user. The input is location information, and the output is a voice announcement of surrounding disaster information. For example, the announcement might say, "Heavy rain is causing flood damage in this area. Please evacuate to higher ground."

[1531] This explains the process by which the system of the present invention provides rapid and accurate information while clarifying the specific operations at each processing step.

[1532] (Application example 1)

[1533] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1534] In modern society, there is a demand for fast and accurate information in the event of a disaster. However, with conventional methods, the collection, analysis, and distribution of disaster information are not centralized, which creates the risk of erroneous information spreading. Furthermore, it is difficult to provide individual disaster information based on real-time location information. This can lead to delays in evacuation behavior and the spread of anxiety-inducing information.

[1535] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1536] In this invention, the server includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating appropriate information, means for converting the generated information into voice and outputting it, means for communicating with other bases through voice, means for collecting information during a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, means for communicating with a central server to acquire and analyze disaster information, means for associating the acquired location information with the disaster information and notifying the user of the association by voice, a communication module for acquiring data from the server, and means for providing disaster information in real time using a voice assistant, thereby enabling accurate and prompt information provision during a disaster.

[1537] A "means for accepting voice input" is a device or software for recognizing and capturing voice instructions from a user.

[1538] "Means for converting voice input into text data" refers to technology for analyzing received voice data and converting it into text information.

[1539] The "means for analyzing text data and generating corresponding information" is a function for processing the converted text data and extracting and generating the necessary information.

[1540] The "means for converting the generated information into voice and outputting it" is a system for converting the generated information into synthesized voice and letting the user hear it.

[1541] "Communication means for performing voice communication with other locations" refers to a network function for exchanging voice data between multiple locations.

[1542] The "information gathering means in the event of a disaster" is a device or system for acquiring relevant information in the event of a disaster.

[1543] "Means for analyzing collected information and selecting applicable information" refers to the function of examining collected data and selecting useful information.

[1544] "Means for obtaining current location information" refers to technology for identifying the current location of a user or device.

[1545] "Means for regularly updating information about disasters in the surrounding area based on the current location" is a function that continuously reflects the latest information about disasters in the surrounding area based on the acquired location information.

[1546] "Means for communicating with a central server to obtain and analyze disaster information" refers to a system for obtaining disaster information from a central database and reviewing and processing that information.

[1547] "Means of associating acquired location information with disaster information and notifying the user as voice" refers to a function that compares location data with disaster information and notifies the user of the generated information as voice.

[1548] A "communication module for acquiring data from a server" is a device for acquiring information from a server via a network.

[1549] "Means of providing disaster information in real time using voice assistants" is a system that uses AI technology to provide disaster information instantly via voice.

[1550] This invention is a smart assistant system that accepts voice input, analyzes it, and outputs voice. Its purpose is to provide accurate information in real time, especially during disasters. The system works in conjunction with several main modules.

[1551] A user uses a smartphone to provide voice input. The smartphone uses a microphone to capture the speech and converts it into text data using speech recognition software (e.g., the speech_recognition library). The converted text data is then analyzed (e.g., using natural language processing techniques) to generate the requested information.

[1552] Next, the smartphone sends the acquired location information (e.g., using the geopy library to acquire GPS data) to the server, which then collects disaster information from a central database (e.g., using the requests library to acquire data from an API), analyzes the collected information using a generative AI model (e.g., a natural language generation model), and selects applicable information.

[1553] The server generates reliable information after analysis and sends it to the smartphone, which receives the data and announces the information to the user as voice using speech synthesis technology (e.g., the pyttsx3 library).

[1554] As a concrete example, suppose a user speaks to their smartphone, saying, "Please tell me the disaster information for my current location." In this case, the voice input is converted into text, and "disaster information" is recognized as the trigger word. The smartphone acquires the current location information and sends it to the server. The server collects and analyzes the latest disaster information around the current location, and generates accurate information such as, "An earthquake with a seismic intensity of 6 has occurred. Please evacuate immediately." The smartphone then announces this information to the user as voice.

[1555] Using generative AI models, it is possible to filter out misinformation and provide reliable information. An example prompt is as follows:

[1556] Please create the following Python code that can handle speech recognition and speech synthesis. The code will retrieve the latest disaster information related to the user's current location from the server and announce it by voice. The input speech is in Japanese.

[1557] In this way, the present invention makes it possible to provide accurate and prompt information in the event of a disaster, thereby ensuring the safety of users.

[1558] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1559] Step 1:

[1560] The user inputs voice commands using the smartphone's microphone, including commands such as "Please tell me disaster information for my current location."

[1561] Step 2:

[1562] The device accepts voice input and converts the voice into text data using voice recognition technology (e.g., the speech_recognition library). The input here is the user's voice data, and the output is a command in text format. Specifically, the device captures the user's voice from the microphone and converts it into text.

[1563] Step 3:

[1564] The terminal analyzes the converted text data and extracts the information that needs to be handled. For example, "disaster information" is recognized as a command word. The input is text data, and the output is a command word based on the analysis results. Specifically, the terminal performs keyword extraction and command analysis of the text.

[1565] Step 4:

[1566] The device uses a GPS module to obtain current location information. The input is GPS data, and the output is the current latitude and longitude information. Specifically, the device obtains the coordinates of the current location from the GPS sensor.

[1567] Step 5:

[1568] The device sends the acquired location information to a central server. The input is location data, and the output is communication packets such as HTTP requests. Specifically, the data is sent to the server via the Internet.

[1569] Step 6:

[1570] The server collects the latest disaster information based on the received location information. The collection source is television, radio, social media APIs, etc. The input is location information, and the output is the collected disaster information. Specifically, it makes an API call and retrieves the corresponding data.

[1571] Step 7:

[1572] The server analyzes the collected disaster information using a generative AI model, filtering out false information and selecting reliable information. The input is raw data, and the output is reliable information as the result of analysis. Specifically, the server performs data analysis and filtering using an AI model.

[1573] Step 8:

[1574] The server sends the analyzed information to the terminal. The input is the analysis result data, and the output is a communication packet. Specifically, the analyzed data is sent as an HTTP response.

[1575] Step 9:

[1576] The device receives the information from the server and announces it to the user as voice using speech synthesis technology (e.g., the pyttsx3 library). The input is the analysis result data, and the output is voice data. Specifically, the device converts text into voice and outputs it from the speaker.

[1577] Step 10:

[1578] The user receives voice information provided by the device and takes appropriate action in the event of a disaster. The input is voice data, and the output is the user's actions. Specifically, the user takes evacuation action by following the voice instructions from the device.

[1579] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1580] The present invention is a system that includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating information to be responded to, means for converting the generated information into voice and outputting it, means for communicating with other bases through voice, means for collecting information in the event of a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating information about disasters in the surrounding area based on the current location, and an emotion engine that recognizes the user's emotions. This system enables the rapid and accurate provision of information between local government facilities and to local residents both during normal times and during disasters.

[1581] System Configuration

[1582] The system includes the following main modules:

[1583] 1. Smart speaker device

[1584] 2. Central Server

[1585] 3. Multimodal AI Module

[1586] 4. Emotion Engine

[1587] Program processing

[1588] 1. Reception duties during normal times

[1589] Voice input and analysis

[1590] The user speaks the reason for their visit into the smart speaker device. The device accepts the voice input and converts the voice into text data using a speech recognition module. The converted text data is then analyzed to understand the reason for the visit.

[1591] emotion recognition

[1592] The emotion engine analyzes the user's voice data and recognizes the user's emotional state.

[1593] Response and guidance

[1594] Based on the analysis and emotion recognition results, the device generates a text response, adjusting the content and tone of the response depending on the results of the emotion engine.

[1595] For example, if the user is nervous, the response may be provided in a relaxing tone.

[1596] Example response: "We will direct you to the appropriate department."

[1597] Speech synthesis

[1598] The generated text response is converted into speech and played to the user in real time.

[1599] 2. Audio conferences during normal times

[1600] Starting an Audio Conference

[1601] The user speaks to the terminal, "Please start the audio conference." The voice is converted into text data, and the data instructing the central server to start the audio conference is sent.

[1602] Processing on the server

[1603] The server sets up an audio conference based on the instructions, obtains the necessary connection information, and sends it to the terminal. The terminal then prompts the necessary locations to join the audio conference.

[1604] Audio conference call

[1605] The terminals at the participating locations start an audio communication session, and the users conduct an audio conference.

[1606] 3. Disaster information dissemination during disasters

[1607] Collection of information

[1608] The server collects disaster information from television, radio, social media, etc. The collected information is analyzed using a generative AI module to eliminate false information.

[1609] Producing accurate information

[1610] After analysis, reliable information is generated and sent to the terminal as voice data.

[1611] Information voice announcement

[1612] The device processes the received disaster information using a voice synthesis system and announces it to the user.

[1613] For example, an announcement may be made such as, "An earthquake of magnitude 7 has occurred. Beware of tsunamis."

[1614] Announcement adjustment by emotion recognition

[1615] emotion recognition

[1616] The emotion engine recognizes the user's emotional state and assesses the user's stress level, especially in times of disaster.

[1617] Response Adjustment

[1618] Based on the results of the emotion engine, the content of the voice notification is adjusted to provide information in a tone and content that reduces the user's stress.

[1619] For example, if a user is in a panic, the announcement will be something like, "Please stay calm. We will provide you with the current information."

[1620] Announcement of local information

[1621] Collection and transmission of location information

[1622] The device uses its built-in GPS module to obtain its current location and transmit it to a central server.

[1623] Generate surrounding information

[1624] The server collects and analyzes information about the surrounding area based on location information, and selects and generates important information. The generated information is then sent to the device as voice data.

[1625] Regular updates and announcements

[1626] The terminal periodically announces updated information to the user using voice synthesis technology.

[1627] For example, an announcement may be made such as, "Heavy rain is causing flooding in this area. Please evacuate to higher ground."

[1628] Information sharing between bases

[1629] Emergency meeting begins

[1630] The user speaks to the terminal, "Please start an emergency meeting." The voice is converted into text data, and the data instructing the central server to start an emergency meeting is sent.

[1631] Processing on the server

[1632] The server sets up an emergency meeting, obtains the necessary connection information, and sends it to the terminal. The terminal then prompts the necessary locations to join the emergency meeting.

[1633] Emergency meeting proceedings

[1634] The terminals at the participating locations start a voice communication session, and the users hold an emergency conference.

[1635] As a result, the smart speaker system of the present invention can provide prompt and accurate information to local government facilities and local residents both during normal times and during disasters. Furthermore, the introduction of an emotion engine enables flexible responses according to the user's psychological state, helping to reduce panic, especially during disasters.

[1636] The processing flow will be explained below.

[1637] Reception duties during normal times

[1638] Step 1:

[1639] The user speaks into the smart speaker device about the reason for their visit.

[1640] For example: "I'm here to see the mayor."

[1641] Step 2:

[1642] The device accepts voice input and uses a speech recognition module to convert the user's speech into text data.

[1643] Speech to text: "I'm here to see the mayor."

[1644] Step 3:

[1645] The device analyzes the text data, determines the reason for the visit, and determines the appropriate response.

[1646] Analyze reason for visit: "Visiting the Mayor"

[1647] Step 4:

[1648] The device simultaneously sends the voice data to an emotion engine to recognize the user's emotions, for example, whether they are nervous.

[1649] Step 5:

[1650] The device generates a text response based on the analysis and emotion recognition results, adjusting the content and tone of the response depending on the results of the emotion engine.

[1651] Produced response: "I'll connect you to the right person." Plus, a relaxing tone

[1652] Step 6:

[1653] The terminal converts the generated text response into speech and plays it to the user in real time.

[1654] Audio conference during normal times

[1655] Step 1:

[1656] The user speaks to the terminal, "Please start the audio conference."

[1657] Step 2:

[1658] The terminal converts the voice into text data.

[1659] Voice-to-text: "Start your audio conference."

[1660] Step 3:

[1661] The terminal sends text data to the central server to instruct the start of the audio conference.

[1662] Step 4:

[1663] The server receives the instructions, sets up the audio conference, obtains the necessary connection information, and sends it to the terminal.

[1664] Acquiring and sending destination information

[1665] Step 5:

[1666] The terminal prompts the necessary locations to join the audio conference.

[1667] Step 6:

[1668] The terminal sends the user's voice to an emotion engine to recognize emotions during the meeting.

[1669] Step 7:

[1670] The server provides information to support the progress of the conference based on the results of the emotion engine, as needed.

[1671] Step 8:

[1672] The users hold audio conferences at the designated terminals.

[1673] Disaster information dissemination during disasters

[1674] Step 1:

[1675] The server collects disaster information from television, radio, and social media.

[1676] Step 2:

[1677] The server analyzes the collected information using a generative AI module and eliminates false information.

[1678] Step 3:

[1679] The server generates reliable and accurate information, converts it into voice data, and transmits it to the terminal.

[1680] Step 4:

[1681] The server transmits disaster information to each terminal and instructs it to make voice announcements.

[1682] Step 5:

[1683] The device processes the transmitted disaster information using a voice synthesis system and checks the user's emotional state using an emotion engine.

[1684] Step 6:

[1685] The terminal adjusts the announcement depending on the user's emotional state.

[1686] Example: "There has been a magnitude 7 earthquake. Beware of tsunamis." (Calm tone)

[1687] Step 7:

[1688] The user receives disaster information from the terminal.

[1689] Announcement of local information

[1690] Step 1:

[1691] The device obtains its current location using the built-in GPS module.

[1692] Step 2:

[1693] The terminal transmits the acquired location information to a central server.

[1694] Location: "Your current location is XXX, YYY."

[1695] Step 3:

[1696] The server collects and analyzes information about surrounding disasters based on location information.

[1697] Step 4:

[1698] The server analyzes the collected peripheral information and selects and generates important information.

[1699] Step 5:

[1700] The server transmits the generated information to each terminal and instructs it to make a voice announcement.

[1701] Step 6:

[1702] The terminal synthesizes the received information into voice and uses an emotion engine to recognize the user's emotional state.

[1703] Step 7:

[1704] The terminal adjusts the announcements according to the emotional state and provides information to the user.

[1705] For example: "Heavy rainfall is causing flooding in this area. Please seek refuge on higher ground." (Reassuring tone)

[1706] Information sharing between bases

[1707] Schedule an emergency meeting

[1708] Step 1:

[1709] The user speaks to the terminal, "Please start an emergency meeting."

[1710] Step 2:

[1711] The terminal converts the voice into text data.

[1712] Step 3:

[1713] The terminal transmits text data to the central server, instructing the start of an emergency meeting.

[1714] Step 4:

[1715] The server sets up an emergency meeting, obtains the necessary connection information, and transmits it to the terminal.

[1716] Step 5:

[1717] The terminal prompts necessary locations to participate in the emergency meeting.

[1718] Step 6:

[1719] The terminal sends the user's voice to an emotion engine to recognize emotions during an emergency meeting.

[1720] Step 7:

[1721] The server provides information to assist in the progress of the emergency meeting, if necessary, based on the results of the emotion engine.

[1722] Step 8:

[1723] The user holds an emergency meeting at the designated terminal.

[1724] In this way, the smart speaker system of the present invention can provide prompt and accurate information to local government facilities and local residents both during normal times and during disasters.Furthermore, the introduction of an emotion engine makes it possible to respond flexibly to the user's emotional state, which is a distinctive feature of the system, reducing the user's psychological burden.

[1725] Example 2

[1726] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1727] While systems using conventional voice recognition technology are required to provide accurate and prompt emergency information during disasters, in reality, they face challenges in effectively communicating information due to the inability to properly assess the user's stress level and the inclusion of false information. Furthermore, in normal reception work and voice conferences, it is difficult to respond according to the user's emotional state, leading to a demand for improved user experience.

[1728] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating information to be handled, means for converting the generated information into voice and outputting it, communication means for performing voice communication with other bases, means for collecting information in the event of a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, and means for recognizing the user's emotional state and adjusting the response content. This makes it possible to quickly provide accurate and appropriate information in the event of a disaster, and to flexibly respond to the user's emotional state in normal times.

[1729] The "means for accepting voice input" is a device that has the function of capturing the voice spoken by the user as a digital signal.

[1730] "Means for converting voice input into text data" refers to technology for analyzing voice signals and converting them into corresponding text data.

[1731] "Means for analyzing text data and generating corresponding information" refers to technology that processes acquired text data and extracts and generates information based on the user's requests and intentions.

[1732] The "means for converting the generated information into voice and outputting it" is a device that has the function of converting text data into a voice signal using voice synthesis technology and playing it aloud to the user.

[1733] "Communication means for performing voice communication with other locations" refers to communication technology for transmitting and receiving voice data between multiple terminals and servers.

[1734] "Means of collecting information during disasters" refers to technology that has the function of collecting disaster-related information from various sources such as television, radio, and social media.

[1735] "Means for analyzing collected information and selecting applicable information" refers to the technology of processing collected information using data analysis technology and selecting highly reliable information.

[1736] "Means for obtaining current location information" refers to technology for identifying the current location using a GPS module or the like.

[1737] "Means for periodically updating surrounding disaster information based on the current location" refers to technology that continuously collects disaster information for the surrounding area based on the acquired location information and updates it to the latest information.

[1738] "Means for recognizing the user's emotional state and adjusting the response content" refers to technology that analyzes emotions from the user's voice and adjusts the response content and tone based on the results.

[1739] MODE FOR CARRYING OUT THE INVENTION

[1740] The present invention includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating corresponding information, means for converting the generated information into voice and outputting it, communication means for performing voice communication with other bases, means for collecting information in the event of a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, and means for recognizing the user's emotional state and adjusting the response content.

[1741] The system of the present invention includes the following main modules:

[1742] 1. A means of accepting voice input (smart speaker device)

[1743] Smart speaker devices have built-in microphones that accept voice input from users. For example, if you say, "I want to go to the government office," the voice is captured by the device as a digital signal.

[1744] 2. A means of converting voice input into text data

[1745] The device converts the received voice input into text using a speech recognition module such as Google Cloud Speech-to-Text, which analyzes the voice signal and converts it into text.

[1746] 3. A method for analyzing text data and generating relevant information

[1747] The converted text data is analyzed using natural language processing (NLP) technology to extract information based on the user's intent and requests. For example, if the user enters "I want to go to the government office," the system identifies "visiting the government office" as the reason for the visit.

[1748] 4. A means of converting the generated information into audio and outputting it

[1749] The text response generated based on the analysis results is converted into speech using a speech synthesis system such as Amazon Polly. The device then plays this speech back to the user, responding, for example, in a relaxing tone, saying, "We will direct you to the appropriate department."

[1750] 5. Communication means for voice communication with other locations

[1751] To enable voice communication between the device and other locations, a connection is set up using the Zoom API for each location, allowing users to hold voice conferences and communicate through the device.

[1752] 6. Information gathering methods during disasters

[1753] The server collects disaster information from sources such as television, radio, and social media. The collected information is then analyzed using OpenAI's generative AI model to filter out unreliable information. This process ensures accurate and prompt information provision.

[1754] 7. How we analyze the collected information and select applicable information

[1755] The collected disaster information is processed using data analysis technology to extract applicable and reliable information, which is then generated in text format and transmitted to the device as voice data.

[1756] 8. How to get current location information

[1757] The device uses its built-in GPS module to determine its current location and sends it to the server using the Google Maps API.

[1758] 9. A way to periodically update information about nearby disasters based on your current location

[1759] The server collects and analyzes disaster information from the surrounding area based on the acquired location information, and selects important information. This information is updated periodically and sent to the device as voice data.

[1760] 10. A way to recognize the user's emotional state and tailor responses

[1761] The emotion engine analyzes the user's voice data and recognizes the user's emotional state using tools such as IBM Watson Tone Analyzer. Particularly in times of disaster, the system assesses the user's stress level and panic and adjusts the content and tone of the response accordingly. For example, the response may be something like, "Please stay calm. We'll provide you with the latest information."

[1762] Examples of concrete examples and prompts

[1763] For example, if a user says to a smart speaker, "I want to go to the government office," the device converts the voice into text data and analyzes the reason for the visit. If the emotion engine determines that the user is nervous, it responds in a relaxing tone, saying, "I will guide you to the relevant department," and plays the response aloud.

[1764] An example prompt is:

[1765] The system converts what the user says into text and analyzes that text to understand the reason for the visit. It then analyzes the user's emotional state and responds by voice, adjusting the tone of the response as needed. For example, if the user says, "I'd like to go to the government office," the system responds in a relaxing tone, saying, "I'll take you to the appropriate department."

[1766] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1767] Processing Steps

[1768] Step 1: Accepting voice input

[1769] Subject: Terminal

[1770] Specific behavior:

[1771] The user speaks into the smart speaker device and says, "I would like to go to the city hall."

[1772] The device captures audio using its built-in microphone.

[1773] input:

[1774] User Voice

[1775] Data processing:

[1776] Converts audio signals into digital data.

[1777] output:

[1778] Digital audio data

[1779] Step 2: Convert audio data to text

[1780] Subject: Terminal

[1781] Specific behavior:

[1782] The device converts the voice data into text data using the Google Cloud Speech-to-Text module.

[1783] input:

[1784] Digital audio data

[1785] Data processing:

[1786] The voice signal is analyzed and converted into corresponding text data.

[1787] output:

[1788] Converted text data (e.g., "I would like to go to the government office.")

[1789] Step 3: Analyzing the text data

[1790] Subject: Terminal

[1791] Specific behavior:

[1792] The device uses natural language processing (NLP) technology to analyze the text data and understand the reason for the visit.

[1793] input:

[1794] Converted text data

[1795] Data processing:

[1796] Through text data analysis, the reason for the visit is identified (e.g., "visiting government office").

[1797] output:

[1798] Analyzed information (e.g., "visit to government office")

[1799] Step 4: Emotion Recognition

[1800] Subject: Terminal

[1801] Specific behavior:

[1802] The emotion engine uses IBM Watson Tone Analyzer to analyze the user's voice data and recognize their emotional state.

[1803] input:

[1804] Converted text data

[1805] Captured audio data

[1806] Data processing:

[1807] Perform tone and sentiment analysis on voice and text data.

[1808] output:

[1809] Emotional state analysis results (e.g., tension)

[1810] Step 5: Response generation and adjustment

[1811] Subject: Terminal

[1812] Specific behavior:

[1813] Based on the analysis and emotion recognition results, the device generates a text response.

[1814] Adjust the content and tone of your response depending on the results of the emotion engine.

[1815] input:

[1816] Analyzed information (e.g., "visit to government office")

[1817] Emotional state analysis results (e.g., tension)

[1818] Data processing:

[1819] Generate text responses and adjust tone

[1820] output:

[1821] Tailored text responses (e.g., "We'll direct you to the appropriate department")

[1822] Step 6: Speech synthesis and response playback

[1823] Subject: Terminal

[1824] Specific behavior:

[1825] The device converts the generated text response into speech using a speech synthesis system such as Amazon Polly.

[1826] Plays a sound to the user.

[1827] input:

[1828] Tailored text responses (e.g., "We'll direct you to the appropriate department")

[1829] Data processing:

[1830] Convert text data into audio signals.

[1831] output:

[1832] Voice response (e.g., "We will direct you to the appropriate department")

[1833] Step 7: Collect disaster information

[1834] Subject: Server

[1835] Specific behavior:

[1836] The server collects disaster information from television, radio, social media, etc.

[1837] input:

[1838] Disaster-related information data from various sources

[1839] Data processing:

[1840] Disaster information collection and data conversion

[1841] output:

[1842] Collected disaster information data

[1843] Step 8: Analysis and selection of disaster information

[1844] Subject: Server

[1845] Specific behavior:

[1846] The server uses OpenAI's generative AI model to analyze the collected disaster information and eliminate false information.

[1847] Select reliable disaster information.

[1848] input:

[1849] Collected disaster information data

[1850] Data processing:

[1851] Data analysis and information selection

[1852] output:

[1853] Reliable disaster information

[1854] Step 9: Obtaining and sending location information

[1855] Subject: Terminal

[1856] Specific behavior:

[1857] The device uses the built-in GPS module to obtain its current location.

[1858] The acquired location information is sent to the server.

[1859] input:

[1860] Location data from the GPS module

[1861] Data processing:

[1862] Acquiring location information and improving data formats

[1863] output:

[1864] Current location information

[1865] Step 10: Collect and generate information on surrounding disasters

[1866] Subject: Server

[1867] Specific behavior:

[1868] The server collects and analyzes information about surrounding disasters based on location information.

[1869] input:

[1870] Current location information

[1871] Data processing:

[1872] Collection and analysis of information on nearby disasters

[1873] output:

[1874] Generated disaster information data

[1875] Step 11: Disaster Information Updates and Announcements

[1876] Subject: Terminal

[1877] Specific behavior:

[1878] The disaster information received by the device is converted into speech using a speech synthesis system such as Amazon Polly.

[1879] Users will be notified of the latest disaster information on a regular basis.

[1880] input:

[1881] Generated disaster information data

[1882] Data processing:

[1883] Audio data generation and announcement of disaster information

[1884] output:

[1885] Voice announcements (e.g., "Heavy rain is causing flooding in this area. Please evacuate to higher ground.")

[1886] Step 12: Recognize the user's emotional state and adjust your response

[1887] Subject: Terminal

[1888] Specific behavior:

[1889] The emotion engine analyzes the user's voice data and assesses their emotional state.

[1890] Tailor your response and tone based on the user's stress level or panic.

[1891] input:

[1892] User voice data

[1893] Emotional state analysis results

[1894] Data processing:

[1895] Emotional state assessment and response adjustment

[1896] output:

[1897] Tailored voice responses (e.g., "Please remain calm. We'll provide you with the latest information.")

[1898] As a result, the present invention enables accurate and prompt provision of information in the event of a disaster, and realizes flexible responses according to the user's emotional state during normal times.

[1899] (Application example 2)

[1900] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1901] Conventional disaster information systems have problems with insufficient real-time information provision and inability to provide information that takes into account the user's emotional state. This makes it difficult for users to take appropriate action in an emergency, increasing the likelihood of them falling into a state of panic. Furthermore, the provision of individual disaster information based on location information is insufficient, making it impossible to encourage appropriate responses based on information specific to the area. It is desirable to solve these problems and ensure the safety of users.

[1902] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating information to be addressed, means for converting the generated information into voice and outputting it, means for communicating with other locations via voice, means for collecting information during a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, emotion analysis means for recognizing emotions, and means for adjusting the content and tone of a voice response based on the emotional state recognized by the emotion analysis means. This enables accurate information to be provided in real time during disasters and emergencies, enabling flexible responses according to the user's emotional state. Furthermore, based on individual location information, disaster information specific to the region can be provided, encouraging appropriate action.

[1903] "Means for accepting voice input" refers to a device or software that has the function of detecting voice and capturing it as a digital signal.

[1904] A "means for converting speech input into text data" is a system or algorithm that analyzes received speech signals and converts them into corresponding text information.

[1905] The "means for analyzing text data and generating corresponding information" refers to a process or module that analyzes input character information and generates information to be provided to the user based on the content of that information.

[1906] The "means for converting generated information into audio and outputting it" is a technology for converting information generated on a text basis into audio and providing it to the user as audio.

[1907] "Communication means for performing voice communication with other locations" refers to communication technology and equipment for transmitting and receiving voice data between devices and systems located in different locations.

[1908] "Means of collecting information during disasters" refers to mechanisms and means for collecting useful information from outside when a disaster occurs.

[1909] "Means for analyzing collected information and selecting applicable information" refers to a system or algorithm for analyzing the various collected information and selecting information that is reliable and useful to the user.

[1910] "Means for obtaining current location information" refers to technology for measuring and obtaining the current location of the device (e.g., GPS module).

[1911] "Means for periodically updating surrounding disaster information based on current location" is a mechanism that uses acquired location information to periodically acquire and update surrounding disaster information.

[1912] "Emotion analysis means for recognizing emotions" refers to technologies and algorithms that analyze a user's voice or text and evaluate and recognize their emotional state.

[1913] The "means for adjusting the content and tone of the voice response based on the emotional state recognized by the emotion analysis means" is a function for appropriately adjusting the content and tone of the generated voice response according to the recognized emotional state of the user.

[1914] The present invention is a system for providing information during disasters and emergencies, and is composed of the following components: First, a smartphone, smart speaker, or other voice-enabled device is used as a means for users to input information by voice, allowing users to request information in real time.

[1915] To convert voice input into text data, we use the Google Speech-to-Text API. This API quickly converts voice data into text data. The text data is then analyzed and the corresponding information is generated. The software used here is the NLTK library, a Python natural language processing library. This library is used to analyze the user's input and generate the corresponding information.

[1916] The generated text data is converted into audio and provided to the user. Using the Google Text-to-Speech API, the generated text data is converted into high-quality audio. The audio output is provided through the speaker of a smartphone or smart speaker.

[1917] This system also has a communication means for voice communication with other locations. For example, when a user starts an audio conference, they can connect to other locations through a calling service such as Google Meet or Zoom and conduct voice communication.

[1918] In the event of a disaster, information is collected from television, radio, social media, etc., and analyzed to select applicable information. A generative AI model is used to analyze the collected information. This model eliminates false information and provides reliable information.

[1919] To obtain current location information, the built-in GPS module of the smartphone is used. Based on the obtained location information, information on disasters in the surrounding area is periodically updated and provided to the user. For example, heavy rain warnings and earthquake information for the area can be collected from the location information and notified to the user.

[1920] The IBM Watson Tone Analyzer is used to analyze emotions. This analyzes the user's voice and text data to recognize their emotional state. Based on the recognized emotional state, the content and tone of the voice response are adjusted. For example, if the user is nervous, information is provided in a slower, calmer tone.

[1921] Specific examples

[1922] A specific example is shown below. When a user says, "Please tell me the latest disaster information for this area," this speech is accepted as voice input and converted into text data using the Google Speech-to-Text API. The text data is then analyzed using the NLTK library, and the relevant disaster information is generated. The generated information is then converted into audio using the Google Text-to-Speech API and provided to the user in real time.

[1923] Example prompt sentence:

[1924] User says: "What are the latest disaster updates for this area?"

[1925] App response: Obtains location information, analyzes the user's emotional state with an emotion engine, and outputs collected disaster information in voice with adjusted tone.

[1926] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1927] Step 1:

[1928] The user inputs information by voice. The user speaks to a smartphone or smart speaker, saying something like, "Please tell me the latest disaster information for this area." The input data is voice data.

[1929] Step 2:

[1930] The device accepts voice input, records the audio as a digital signal, and then uses the Google Speech-to-Text API to convert this audio data into text data. The input is audio data, and the output is audio as text data.

[1931] Step 3:

[1932] The server receives the text data and analyzes it using Python's NLTK library. This analysis extracts important keywords and phrases from the input text. The data processing performed here involves analyzing the text and extracting the necessary information. The output is the analyzed text data.

[1933] Step 4:

[1934] The server generates appropriate disaster information based on the extracted keywords and phrases. Disaster information is collected using, for example, a disaster prevention information API. The generated information is expressed in text format. The input is analyzed text data, and the output is disaster information text data.

[1935] Step 5:

[1936] The server uses a generative AI model to analyze the collected disaster information and select applicable information. The generative AI model evaluates the reliability of the collected information and filters out unnecessary information. The input is text data of disaster information, and the output is reliable disaster information.

[1937] Step 6:

[1938] The server converts text data into audio data using the Google Text-to-Speech API to output the selected disaster information as audio. The input is reliable disaster information text data, and the output is audio data.

[1939] Step 7:

[1940] The voice data is provided to the user through the speaker of a smartphone or smart speaker. Conveniently, the generated voice is delivered to the user in real time. The output voice is then listened to by the user.

[1941] Step 8:

[1942] As a means of emotion analysis, IBM Watson Tone Analyzer is used to analyze the user's voice or text data to recognize their emotional state. The input is the user's voice or text data, and the output is evaluation data of the user's emotional state.

[1943] Step 9:

[1944] The server adjusts the content and tone of the voice response based on the emotion analysis data. The speed and tone of the voice change depending on the user's emotional state, reducing stress and panic in the user. The input is emotion analysis data, and the output is adjusted voice data.

[1945] For example:

[1946] When a user voice-inputs "I want disaster information," the speech is converted into text data by the Google Speech-to-Text API and analyzed by the Python NLTK library. The latest disaster information obtained from the disaster prevention information API is then converted into audio data by the Google Text-to-Speech API and provided to the user through the smartphone speaker. At the same time, the IBM Watson Tone Analyzer evaluates the user's emotional state and provides information in an appropriate tone.

[1947] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1948] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1949] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1950] [Fourth embodiment]

[1951] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1952] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1953] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1954] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1955] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1956] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1957] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1958] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1959] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1960] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1961] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1962] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1963] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1964] The present invention is a system including means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating information to be responded to, means for converting the generated information into voice and outputting it, means for communicating with other bases through voice, means for collecting information in the event of a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, and means for periodically updating surrounding disaster information based on the current location. This system enables the rapid and accurate provision of information between local government facilities and to local residents both during normal times and during disasters.

[1965] System Configuration

[1966] The system includes the following main modules:

[1967] 1. Smart speaker device

[1968] 2. Central Server

[1969] 3. Multimodal AI Module

[1970] Program processing

[1971] 1. Reception duties during normal times

[1972] Voice input and analysis

[1973] The user speaks the reason for their visit into the smart speaker device. The device accepts the voice input and converts the voice into text data using a speech recognition module. The converted text data is then analyzed to understand the reason for the visit.

[1974] For example, if a user says, "I'm here to see the mayor," the device will parse this text and generate directions to the appropriate department.

[1975] Response and guidance

[1976] Based on the analysis results, the terminal generates a text response and plays it back to the user in real time using speech synthesis technology.

[1977] As an example of a response, a voice saying "We will direct you to the relevant department" is played.

[1978] 2. Audio conferences during normal times

[1979] Starting an Audio Conference

[1980] The user speaks to the terminal, "Please start the audio conference." The terminal converts the speech into text and sends data to the central server instructing it to start the audio conference.

[1981] Processing on the server

[1982] The server sets up an audio conference based on the received instructions, obtains the necessary connection information, and sends it to the terminal. The terminal then prompts the necessary locations to join the audio conference.

[1983] Audio conference call

[1984] The terminals at the participating locations start an audio communication session, and the users conduct an audio conference.

[1985] 3. Disaster information dissemination during disasters

[1986] Collection of information

[1987] The server collects disaster information from television, radio, social media, etc. The collected information is analyzed using a generative AI module, and false information is filtered out.

[1988] Producing accurate information

[1989] After analysis, reliable information is generated and sent to the terminal as voice data.

[1990] Information voice announcement

[1991] The terminal announces the received disaster information to the user using voice synthesis technology.

[1992] For example, an announcement may be made such as, "An earthquake of magnitude 7 has occurred. Beware of tsunamis."

[1993] 4. Local area information announcements

[1994] Collection and transmission of location information

[1995] The device uses the built-in GPS module to obtain current location information, which is then sent to the server.

[1996] Generate surrounding information

[1997] The server collects and analyzes information about the surrounding area based on location information, and selects and generates important information, which is then sent to the device as voice data.

[1998] Regular updates and announcements

[1999] The terminal periodically announces updated information to the user using voice synthesis technology.

[2000] For example, an announcement may be made saying, "Heavy rain is causing flooding in this area. Please evacuate to higher ground."

[2001] As a result, the smart speaker system of the present invention will be able to provide prompt and accurate information to local government facilities and local residents both during normal times and during disasters. Furthermore, by using a generative AI model, the system is able to eliminate false information and provide highly reliable information.

[2002] The processing flow will be explained below.

[2003] Reception duties during normal times

[2004] Step 1:

[2005] The user speaks into the smart speaker device about the reason for their visit.

[2006] For example: "I'm here to see the mayor."

[2007] Step 2:

[2008] The device accepts voice input and uses a speech recognition module to convert the user's speech into text data.

[2009] Speech to text: "I'm here to see the mayor."

[2010] Step 3:

[2011] The device analyzes the text data and determines the appropriate response based on the reason for the visit.

[2012] Analyze reason for visit: "Visiting the Mayor"

[2013] Step 4:

[2014] The terminal generates a text response based on the results of the analysis.

[2015] Response generation: "We will direct you to the appropriate department."

[2016] Step 5:

[2017] The terminal converts the generated text response into speech and plays it back.

[2018] Text-to-speech: "We will direct you to the appropriate department."

[2019] Audio conference during normal times

[2020] Step 1:

[2021] The user speaks to the terminal, "Please start the audio conference."

[2022] Step 2:

[2023] The terminal converts the voice into text data.

[2024] Voice-to-text: "Start your audio conference."

[2025] Step 3:

[2026] The terminal sends text data to the central server to instruct the start of the audio conference.

[2027] Step 4:

[2028] The server receives the instructions, sets up the audio conference, obtains the necessary connection information, and sends it to the terminal.

[2029] Acquiring and sending destination information

[2030] Step 5:

[2031] The terminal prompts the necessary locations to join the audio conference.

[2032] Step 6:

[2033] The users hold audio conferences at the designated terminals.

[2034] Disaster information dissemination during disasters

[2035] Step 1:

[2036] The server collects disaster information from television, radio, and social media.

[2037] Step 2:

[2038] The server analyzes the collected information using a generative AI module and eliminates false information.

[2039] Step 3:

[2040] The server generates reliable and accurate information and converts it into voice data.

[2041] Step 4:

[2042] The server transmits disaster information to each terminal and instructs it to make voice announcements.

[2043] Step 5:

[2044] The terminal processes the transmitted disaster information using a voice synthesis system and announces it to the user.

[2045] Example announcement: "An earthquake of magnitude 7 has occurred. Beware of tsunamis."

[2046] Announcement of local information

[2047] Step 1:

[2048] The device obtains its current location using the built-in GPS module.

[2049] Step 2:

[2050] The terminal transmits the acquired location information to a central server.

[2051] Location: "Your current location is XXX, YYY."

[2052] Step 3:

[2053] The server collects and analyzes information about surrounding disasters based on location information.

[2054] Step 4:

[2055] The server analyzes the collected peripheral information and selects and generates important information.

[2056] Step 5:

[2057] The server transmits the generated information to each terminal and instructs it to make a voice announcement.

[2058] Step 6:

[2059] The terminal synthesizes the received information into voice and announces it to the user.

[2060] Example announcement: "Heavy rain is causing flooding in this area. Please evacuate to higher ground."

[2061] Information sharing between bases

[2062] Step 1:

[2063] The user speaks to the terminal, "Please start an emergency meeting."

[2064] Step 2:

[2065] The terminal converts the voice into text data.

[2066] Step 3:

[2067] The terminal transmits text data to the central server, instructing the start of an emergency meeting.

[2068] Step 4:

[2069] The server sets up an emergency meeting, acquires connection destination information, and transmits it to the terminal.

[2070] Step 5:

[2071] The terminal prompts necessary locations to participate in the emergency meeting.

[2072] Step 6:

[2073] The user holds an emergency meeting at the designated terminal.

[2074] In this way, the smart speaker system of the present invention can provide quick and accurate information to local government facilities and local residents both during normal times and during disasters.

[2075] Example 1

[2076] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2077] The present invention aims to provide a system that eliminates false information and provides highly reliable information in situations where rapid and accurate information provision is required during disasters. It also aims to improve business efficiency and facilitate the implementation of voice conferences during normal times.

[2078] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2079] In this invention, the server includes means for accepting voice input from a user, means for converting the voice input into text data, means for analyzing the text data and generating information to be addressed, means for converting the generated information into voice and outputting it, means for communicating with other locations by voice, means for collecting information during a disaster, means for analyzing the collected information, eliminating false information, and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, means for generating and selecting the information using a generative AI model, and means including a voice recognition module and a voice synthesis module. This enables rapid and accurate information provision, improved business efficiency, and smooth voice conferences during disasters and normal times.

[2080] A "means for accepting voice input" is a device for detecting and recording voices uttered by a user.

[2081] The "means for converting voice input into text data" refers to software or algorithms for analyzing acquired voice data and converting it into text information.

[2082] "Means for analyzing text data and generating corresponding information" refers to a system that uses converted text data to understand and analyze the user's intentions and requests, and generates appropriate responses and guidance.

[2083] The "means for converting the generated information into voice and outputting it" refers to a voice synthesis technology and its output mechanism for reproducing text information as voice.

[2084] "Communication means for performing voice communication with other locations" refers to technology and systems for performing voice communication with multiple remote locations via a network.

[2085] "Means of collecting information during disasters" refers to systems and technologies for collecting relevant information from various sources when a disaster occurs.

[2086] "Means for analyzing collected information, eliminating false information, and selecting applicable information" refers to technology that evaluates the credibility of collected information, eliminates incorrect information, and selects highly reliable information.

[2087] "Means for obtaining current location information" refers to a system that uses GPS or other location information acquisition technology to determine the device's current location.

[2088] "Means for regularly updating disaster information in the surrounding area based on the current location" refers to a system that continuously obtains information on the disaster situation in the surrounding area based on the acquired location information and provides the latest information.

[2089] "Means for utilizing generative AI models" refers to systems that utilize artificial intelligence technology to generate appropriate information from collected and analyzed data.

[2090] A "voice recognition module" is a technology and software for analyzing voice data and converting it into text data.

[2091] A "voice synthesis module" is a technology and software for converting text data into voice data and playing it back.

[2092] The present invention relates to a system that receives, converts, and analyzes voice input, and converts the generated information into voice for output. Specific ways in which the present invention can be implemented are described below.

[2093] Hardware Configuration

[2094] 1. Smart speaker device:

[2095] A microphone to accept voice input

[2096] Speakers for playing audio

[2097] GPS module for obtaining location information

[2098] 2. Central Server:

[2099] High-performance server for data processing and analysis

[2100] A network interface for communicating with smart speaker devices in various locations

[2101] 3. Communications infrastructure:

[2102] A fast and stable internet connection

[2103] Software Configuration

[2104] 1. Speech Recognition Module:

[2105] Natural language processing (NLP) techniques for converting voice input into text data (e.g., popular cloud-based speech recognition services)

[2106] 2. Generative AI Model:

[2107] Artificial intelligence techniques (e.g., generative AI models) for analyzing text data and generating appropriate responses or information

[2108] 3. Speech synthesis module:

[2109] Speech synthesis technology to convert the generated text data into speech (e.g., a cloud-based speech synthesis service)

[2110] 4. Disaster information collection and analysis system:

[2111] Software that collects and analyzes information from various sources (e.g., television, radio, social media)

[2112] Operation overview

[2113] Reception duties during normal times

[2114] 1. Voice input and analysis:

[2115] The user speaks into the smart speaker device and explains the reason for their visit, for example, "I'm here to see the mayor."

[2116] The device accepts voice input and converts the voice into text data using a voice recognition module.

[2117] The converted text data is sent to a central server for analysis to understand the reason for the visit.

[2118] 2. Responses and Guidance:

[2119] The server generates a response to the user based on the analysis results.

[2120] The device uses a speech synthesis module to play back the response aloud.

[2121] Example: "We will refer you to the appropriate department."

[2122] Audio conference during normal times

[2123] 1. Start an audio conference:

[2124] A user says, "Start audio conference."

[2125] The device converts the speech into text and sends it to a central server.

[2126] The server sets up the audio conference and sends the necessary connection information to the terminal.

[2127] 2. Audio Conference Procedures:

[2128] An audio communication session is initiated between the terminals, and the users are audio conferencing.

[2129] Disaster information dissemination during disasters

[2130] 1. Information Collection and Analysis:

[2131] The server collects disaster information from various sources and uses a generative AI model for analysis to eliminate false information.

[2132] 2. Information Generation and Announcement:

[2133] The server generates reliable information and sends it to the terminal.

[2134] The terminal uses voice synthesis technology to announce information to the user.

[2135] Example: "An earthquake of magnitude 7 has occurred. Beware of tsunamis."

[2136] Announcement of local information

[2137] 1. Obtaining and sending location information:

[2138] The device uses a GPS module to obtain its current location information and transmits it to a central server.

[2139] 2. Peripheral information generation and announcement:

[2140] The server collects and analyzes information about disasters in the surrounding area based on location information, selects important information, and sends it to the terminal.

[2141] The device uses voice synthesis technology to periodically announce information to the user.

[2142] For example: "Heavy rain is causing flooding in this area. Please evacuate to higher ground."

[2143] Specific examples (related to generative AI models)

[2144] Prompt Sentence Examples

[2145] "When a user asks a smart speaker how to respond in the event of a disaster, the system generates prompts to provide reliable information."

[2146] Specific prompt sentences

[2147] "A magnitude 7 earthquake has occurred. Please provide evacuation routes and precautions to users living in areas where a tsunami may occur."

[2148] With the above-described configuration and operation, the smart speaker system of the present invention enables prompt and accurate information provision to local government facilities and local residents both during normal times and in times of disaster.

[2149] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2150] Step 1: Accepting voice input

[2151] The user speaks into the smart speaker device. The device uses a microphone to capture voice data. The input is the user's voice data, and the output is the captured voice data. This voice data is sent to the voice recognition module in the next step.

[2152] Step 2: Convert voice to text

[2153] The terminal sends the acquired voice data to the voice recognition module, which converts the voice data into text data. The input is the acquired voice data, and the output is text data. This text data is analyzed in the next step.

[2154] Step 3: Analyzing the text data

[2155] The terminal sends the converted text data to a central server, which then analyzes the text data using a natural language processing (NLP) module to understand the user's intent. For example, the text data "I've come to see the mayor" is analyzed to determine the appropriate department to respond to the request. The input is text data, and the output is the analysis result.

[2156] Step 4: Generate response text

[2157] The server generates a response to the user based on the analysis results. A generative AI model is used for this generation. The input is the analysis results, and the output is response text data. For example, the generated text might say, "We will direct you to the relevant department."

[2158] Step 5: Respond with text-to-speech

[2159] The terminal sends the response text data received from the server to the speech synthesis module. The speech synthesis module converts the text data into voice data and plays it back to the user through a speaker. The input is the response text data, and the output is voice data. For example, the voice may say, "We will direct you to the relevant department."

[2160] Step 6: Start an audio conference

[2161] The user speaks to the terminal, "Please start the audio conference." The terminal converts the speech into text data and sends it to the central server. The input is the user's voice data, and the output is the instruction text to start the audio conference.

[2162] Step 7: Configure the server for audio conferencing

[2163] The server receives the voice conference start instruction and sets up the voice conference. It obtains the necessary connection information using VOIP technology and sends it to the terminal. The input is the voice conference start instruction text, and the output is the connection information.

[2164] Step 8: Start a voice communication session

[2165] Each terminal starts a voice communication session based on the connection destination information from the server. This allows users to hold a voice conference. The input is the connection destination information, and the output is a voice communication session.

[2166] Step 9: Collect and analyze disaster information

[2167] The server collects disaster information from sources such as television, radio, and social media. It then analyzes the collected information using a generative AI model to eliminate false information. The input is disaster information data, and the output is highly reliable disaster information.

[2168] Step 10: Disaster information generation and announcement

[2169] The server generates highly reliable information after analysis and sends it to the terminal. The terminal converts the received information into voice data using voice synthesis technology and announces it to the user. The input is highly reliable disaster information, and the output is a voice announcement of the disaster information. For example, the announcement might say, "An earthquake with a seismic intensity of 7 has occurred. Beware of tsunamis."

[2170] Step 11: Obtaining and sending location information

[2171] The device acquires its current location information using the built-in GPS module and sends it to the server. The input is the location information, and the output is the data sent to the server.

[2172] Step 12: Generate and announce surrounding information

[2173] The server collects and analyzes information about surrounding disasters based on the received location information. It selects important information and sends it to the terminal, which then uses voice synthesis technology to announce it to the user. The input is location information, and the output is a voice announcement of surrounding disaster information. For example, the announcement might say, "Heavy rain is causing flood damage in this area. Please evacuate to higher ground."

[2174] This explains the process by which the system of the present invention provides rapid and accurate information while clarifying the specific operations at each processing step.

[2175] (Application example 1)

[2176] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2177] In modern society, there is a demand for fast and accurate information in the event of a disaster. However, with conventional methods, the collection, analysis, and distribution of disaster information are not centralized, which creates the risk of erroneous information spreading. Furthermore, it is difficult to provide individual disaster information based on real-time location information. This can lead to delays in evacuation behavior and the spread of anxiety-inducing information.

[2178] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2179] In this invention, the server includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating appropriate information, means for converting the generated information into voice and outputting it, means for communicating with other bases through voice, means for collecting information during a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating surrounding disaster information based on the current location, means for communicating with a central server to acquire and analyze disaster information, means for associating the acquired location information with the disaster information and notifying the user of the association by voice, a communication module for acquiring data from the server, and means for providing disaster information in real time using a voice assistant, thereby enabling accurate and prompt information provision during a disaster.

[2180] A "means for accepting voice input" is a device or software for recognizing and capturing voice instructions from a user.

[2181] "Means for converting voice input into text data" refers to technology for analyzing received voice data and converting it into text information.

[2182] The "means for analyzing text data and generating corresponding information" is a function for processing the converted text data and extracting and generating the necessary information.

[2183] The "means for converting the generated information into voice and outputting it" is a system for converting the generated information into synthesized voice and letting the user hear it.

[2184] "Communication means for performing voice communication with other locations" refers to a network function for exchanging voice data between multiple locations.

[2185] The "information gathering means in the event of a disaster" is a device or system for acquiring relevant information in the event of a disaster.

[2186] "Means for analyzing collected information and selecting applicable information" refers to the function of examining collected data and selecting useful information.

[2187] "Means for obtaining current location information" refers to technology for identifying the current location of a user or device.

[2188] "Means for regularly updating information about disasters in the surrounding area based on the current location" is a function that continuously reflects the latest information about disasters in the surrounding area based on the acquired location information.

[2189] "Means for communicating with a central server to obtain and analyze disaster information" refers to a system for obtaining disaster information from a central database and reviewing and processing that information.

[2190] "Means of associating acquired location information with disaster information and notifying the user as voice" refers to a function that compares location data with disaster information and notifies the user of the generated information as voice.

[2191] A "communication module for acquiring data from a server" is a device for acquiring information from a server via a network.

[2192] "Means of providing disaster information in real time using voice assistants" is a system that uses AI technology to provide disaster information instantly via voice.

[2193] This invention is a smart assistant system that accepts voice input, analyzes it, and outputs voice. Its purpose is to provide accurate information in real time, especially during disasters. The system works in conjunction with several main modules.

[2194] A user uses a smartphone to provide voice input. The smartphone uses a microphone to capture the speech and converts it into text data using speech recognition software (e.g., the speech_recognition library). The converted text data is then analyzed (e.g., using natural language processing techniques) to generate the requested information.

[2195] Next, the smartphone sends the acquired location information (e.g., using the geopy library to acquire GPS data) to the server, which then collects disaster information from a central database (e.g., using the requests library to acquire data from an API), analyzes the collected information using a generative AI model (e.g., a natural language generation model), and selects applicable information.

[2196] The server generates reliable information after analysis and sends it to the smartphone, which receives the data and announces the information to the user as voice using speech synthesis technology (e.g., the pyttsx3 library).

[2197] As a concrete example, suppose a user speaks to their smartphone, saying, "Please tell me the disaster information for my current location." In this case, the voice input is converted into text, and "disaster information" is recognized as the trigger word. The smartphone acquires the current location information and sends it to the server. The server collects and analyzes the latest disaster information around the current location, and generates accurate information such as, "An earthquake with a seismic intensity of 6 has occurred. Please evacuate immediately." The smartphone then announces this information to the user as voice.

[2198] Using generative AI models, it is possible to filter out misinformation and provide reliable information. An example prompt is as follows:

[2199] Please create the following Python code that can handle speech recognition and speech synthesis. The code will retrieve the latest disaster information related to the user's current location from the server and announce it by voice. The input speech is in Japanese.

[2200] In this way, the present invention makes it possible to provide accurate and prompt information in the event of a disaster, thereby ensuring the safety of users.

[2201] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2202] Step 1:

[2203] The user inputs voice commands using the smartphone's microphone, including commands such as "Please tell me disaster information for my current location."

[2204] Step 2:

[2205] The device accepts voice input and converts the voice into text data using voice recognition technology (e.g., the speech_recognition library). The input here is the user's voice data, and the output is a command in text format. Specifically, the device captures the user's voice from the microphone and converts it into text.

[2206] Step 3:

[2207] The terminal analyzes the converted text data and extracts the information that needs to be handled. For example, "disaster information" is recognized as a command word. The input is text data, and the output is a command word based on the analysis results. Specifically, the terminal performs keyword extraction and command analysis of the text.

[2208] Step 4:

[2209] The device uses a GPS module to obtain current location information. The input is GPS data, and the output is the current latitude and longitude information. Specifically, the device obtains the coordinates of the current location from the GPS sensor.

[2210] Step 5:

[2211] The device sends the acquired location information to a central server. The input is location data, and the output is communication packets such as HTTP requests. Specifically, the data is sent to the server via the Internet.

[2212] Step 6:

[2213] The server collects the latest disaster information based on the received location information. The collection source is television, radio, social media APIs, etc. The input is location information, and the output is the collected disaster information. Specifically, it makes an API call and retrieves the corresponding data.

[2214] Step 7:

[2215] The server analyzes the collected disaster information using a generative AI model, filtering out false information and selecting reliable information. The input is raw data, and the output is reliable information as the result of analysis. Specifically, the server performs data analysis and filtering using an AI model.

[2216] Step 8:

[2217] The server sends the analyzed information to the terminal. The input is the analysis result data, and the output is a communication packet. Specifically, the analyzed data is sent as an HTTP response.

[2218] Step 9:

[2219] The device receives the information from the server and announces it to the user as voice using speech synthesis technology (e.g., the pyttsx3 library). The input is the analysis result data, and the output is voice data. Specifically, the device converts text into voice and outputs it from the speaker.

[2220] Step 10:

[2221] The user receives voice information provided by the device and takes appropriate action in the event of a disaster. The input is voice data, and the output is the user's actions. Specifically, the user takes evacuation action by following the voice instructions from the device.

[2222] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2223] The present invention is a system that includes means for accepting voice input, means for converting the voice input into text data, means for analyzing the text data and generating information to be responded to, means for converting the generated information into voice and outputting it, means for communicating with other bases through voice, means for collecting information in the event of a disaster, means for analyzing the collected information and selecting applicable information, means for acquiring current location information, means for periodically updating information about disasters in the surrounding area based on the current location, and an emotion engine that recognizes the user's emotions. This system enables the rapid and accurate provision of information between local government facilities and to local residents both during normal times and during disasters.

[2224] System Configuration

[2225] The system includes the following main modules:

[2226] 1. Smart speaker device

[2227] 2. Central Server

[2228] 3. Multimodal AI Module

[2229] 4. Emotion Engine

[2230] Program processing

[2231] 1. Reception duties during normal times

[2232] Voice input and analysis

[2233] The user speaks the reason for their visit into the smart speaker device. The device accepts the voice input and converts the voice into text data using a speech recognition module. The converted text data is then analyzed to understand the reason for the visit.

[2234] emotion recognition

[2235] The emotion engine analyzes the user's voice data and recognizes the user's emotional state.

[2236] Response and guidance

[2237] Based on the analysis and emotion recognition results, the device generates a text response, adjusting the content and tone of the response depending on the results of the emotion engine.

[2238] For example, if the user is nervous, the response may be provided in a relaxing tone.

[2239] Example response: "We will direct you to the appropriate department."

[2240] Speech synthesis

[2241] The generated text response is converted into speech and played to the user in real time.

[2242] 2. Audio conferences during normal times

[2243] Starting an Audio Conference

[2244] The user speaks to the terminal, "Please start the audio conference." The voice is converted into text data, and the data instructing the central server to start the audio conference is sent.

[2245] Processing on the server

[2246] The server sets up an audio conference based on the instructions, obtains the necessary connection information, and sends it to the terminal. The terminal then prompts the necessary locations to join the audio conference.

[2247] Audio conference call

[2248] The terminals at the participating locations start an audio communication session, and the users conduct an audio conference.

[2249] 3. Disaster information dissemination during disasters

[2250] Collection of information

[2251] The server collects disaster information from television, radio, social media, etc. The collected information is analyzed using a generative AI module to eliminate false information.

[2252] Producing accurate information

[2253] After analysis, reliable information is generated and sent to the terminal as voice data.

[2254] Information voice announcement

[2255] The device processes the received disaster information using a voice synthesis system and announces it to the user.

[2256] For example, an announcement may be made such as, "An earthquake of magnitude 7 has occurred. Beware of tsunamis."

[2257] Announcement adjustment by emotion recognition

[2258] emotion recognition

[2259] The emotion engine recognizes the user's emotional state and assesses the user's stress level, especially in times of disaster.

[2260] Response Adjustment

[2261] Based on the results of the emotion engine, the content of the voice notification is adjusted to provide information in a tone and content that reduces the user's stress.

[2262] For example, if a user is in a panic, the announcement will be something like, "Please stay calm. We will provide you with the current information."

[2263] Announcement of local information

[2264] Collection and transmission of location information

[2265] The device uses its built-in GPS module to obtain its current location and transmit it to a central server.

[2266] Generate surrounding information

[2267] The server collects and analyzes information about the surrounding area based on location information, and selects and generates important information. The generated information is then sent to the device as voice data.

[2268] Regular updates and announcements

[2269] The terminal periodically announces updated information to the user using voice synthesis technology.

[2270] For example, an announcement may be made such as, "Heavy rain is causing flooding in this area. Please evacuate to higher ground."

[2271] Information sharing between bases

[2272] Emergency meeting begins

[2273] The user speaks to the terminal, "Please start an emergency meeting." The voice is converted into text data, and the data instructing the central server to start an emergency meeting is sent.

[2274] Processing on the server

[2275] The server sets up an emergency meeting, obtains the necessary connection information, and sends it to the terminal. The terminal then prompts the necessary locations to join the emergency meeting.

[2276] Emergency meeting proceedings

[2277] The terminals at the participating locations start a voice communication session, and the users hold an emergency conference.

[2278] As a result, the smart speaker system of the present invention can provide prompt and accurate information to local government facilities and local residents both during normal times and during disasters. Furthermore, the introduction of an emotion engine enables flexible responses according to the user's psychological state, helping to reduce panic, especially during disasters.

[2279] The processing flow will be explained below.

[2280] Reception duties during normal times

[2281] Step 1:

[2282] The user speaks into the smart speaker device about the reason for their visit.

[2283] For example: "I'm here to see the mayor."

[2284] Step 2:

[2285] The device accepts voice input and uses a speech recognition module to convert the user's speech into text data.

[2286] Speech to text: "I'm here to see the mayor."

[2287] Step 3:

[2288] The device analyzes the text data, determines the reason for the visit, and determines the appropriate response.

[2289] Analyze reason for visit: "Visiting the Mayor"

[2290] Step 4:

[2291] The device simultaneously sends the voice data to an emotion engine to recognize the user's emotions, for example, whether they are nervous.

[2292] Step 5:

[2293] The device generates a text response based on the analysis and emotion recognition results, adjusting the content and tone of the response depending on the results of the emotion engine.

[2294] Produced response: "I'll connect you to the right person." Plus, a relaxing tone

[2295] Step 6:

[2296] The terminal converts the generated text response into speech and plays it to the user in real time.

[2297] Audio conference during normal times

[2298] Step 1:

[2299] The user speaks to the terminal, "Please start the audio conference."

[2300] Step 2:

[2301] The terminal converts the voice into text data.

[2302] Voice-to-text: "Start your audio conference."

[2303] Step 3:

[2304] The terminal sends text data to the central server to instruct the start of the audio conference.

[2305] Step 4:

[2306] The server receives the instructions, sets up the audio conference, obtains the necessary connection information, and sends it to the terminal.

[2307] Acquiring and sending destination information

[2308] Step 5:

[2309] The terminal prompts the necessary locations to join the audio conference.

[2310] Step 6:

[2311] The terminal sends the user's voice to an emotion engine to recognize emotions during the meeting.

[2312] Step 7:

[2313] The server provides information to support the progress of the conference based on the results of the emotion engine, as needed.

[2314] Step 8:

[2315] The users hold audio conferences at the designated terminals.

[2316] Disaster information dissemination during disasters

[2317] Step 1:

[2318] The server collects disaster information from television, radio, and social media.

[2319] Step 2:

[2320] The server analyzes the collected information using a generative AI module and eliminates false information.

[2321] Step 3:

[2322] The server generates reliable and accurate information, converts it into voice data, and transmits it to the terminal.

[2323] Step 4:

[2324] The server transmits disaster information to each terminal and instructs it to make voice announcements.

[2325] Step 5:

[2326] The device processes the transmitted disaster information using a voice synthesis system and checks the user's emotional state using an emotion engine.

[2327] Step 6:

[2328] The terminal adjusts the announcement depending on the user's emotional state.

[2329] Example: "There has been a magnitude 7 earthquake. Beware of tsunamis." (Calm tone)

[2330] Step 7:

[2331] The user receives disaster information from the terminal.

[2332] Announcement of local information

[2333] Step 1:

[2334] The device obtains its current location using the built-in GPS module.

[2335] Step 2:

[2336] The terminal transmits the acquired location information to a central server.

[2337] Location: "Your current location is XXX, YYY."

[2338] Step 3:

[2339] The server collects and analyzes information about surrounding disasters based on location information.

[2340] Step 4:

[2341] The server analyzes the collected peripheral information and selects and generates important information.

[2342] Step 5:

[2343] The server transmits the generated information to each terminal and instructs it to make a voice announcement.

[2344] Step 6:

[2345] The terminal synthesizes the received information into voice and uses an emotion engine to recognize the user's emotional state.

[2346] Step 7:

[2347] The terminal adjusts the announcements according to the emotional state and provides information to the user.

[2348] For example: "Heavy rainfall is causing flooding in this area. Please seek refuge on higher ground." (Reassuring tone)

[2349] Information sharing between bases

[2350] Schedule an emergency meeting

[2351] Step 1:

[2352] The user speaks to the terminal, "Please start an emergency meeting."

[2353] Step 2:

[2354] The terminal converts the voice into text data.

[2355] Step 3:

[2356] The terminal transmits text data to the central server, instructing the start of an emergency meeting.

[2357] Step 4:

[2358] The server sets up an emergency meeting, obtains the necessary connection information, and transmits it to the terminal.

[2359] Step 5:

[2360] The terminal prompts necessary locations to participate in the emergency meeting.

[2361] Step 6:

[2362] The terminal sends the user's voice to an emotion engine to recognize emotions during an emergency meeting.

[2363] Step 7:

[2364] The server provides information to assist in the progress of the emergency meeting, if necessary, based on the results of the emotion engine.

[2365] Step 8:

[2366] The user holds an emergency meeting at the designated terminal. ...

Claims

1. means for accepting voice input; means for converting voice input into text data; A means for analyzing text data and generating corresponding information; means for converting the generated information into voice and outputting the voice; a communication means for performing voice communication with other locations; Means of collecting information during disasters; A means for analyzing the collected information and selecting applicable information; A means for obtaining current location information; a means for periodically updating information about disasters in the vicinity based on the current location; A system including:

2. 2. The system according to claim 1, wherein the means for accepting voice input is a terminal provided in a local government facility.

3. 10. The system of claim 1, wherein the means for generating and filtering information utilizes a generative AI model.

Images