system

Abstract

The system according to this embodiment aims to provide a means for disaster victims to convey a final message to their loved ones during a disaster. [Solution] The system according to this embodiment comprises an activation unit, a recording unit, a dialogue unit, and a transmission unit. The activation unit is activated automatically in the event of a disaster. The recording unit records and stores the voices and thoughts of disaster victims. The dialogue unit alleviates anxiety through dialogue based on the information recorded by the recording unit. The transmission unit transmits the stored messages.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, there is no sufficient means for disaster victims to convey their last messages to important people, and there is room for improvement.

[0005] The system according to the embodiment aims to provide a means for disaster victims to convey their last messages to important people during a disaster.

Means for Solving the Problems

[0006] The system according to the embodiment includes a startup unit, a recording unit, an interaction unit, and a transmission unit. The startup unit automatically starts during a disaster. The recording unit records and stores the voices and thoughts of disaster victims. The interaction unit alleviates anxiety through interaction based on the information recorded by the recording unit. The transmission unit transmits the saved messages. [Effects of the Invention]

[0007] The system according to this embodiment can provide a means for disaster victims to convey a final message to their loved ones during a disaster. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. Also, the reception device 38, the output device 40, and the camera 42 are connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The disaster message system according to an embodiment of the present invention is a system that delivers a final message to a loved one when facing a life-threatening situation during a disaster. This system is installed on a smartphone and automatically activates during disasters such as tsunamis and landslides, recording and storing the voice and thoughts of the disaster victim. The system uses AI to alleviate anxiety through dialogue and provides a sense of security by supporting those who are feeling lonely. Furthermore, the stored messages are sent to family and friends after safety has been confirmed. This system allows disaster victims to convey their last words and provides emotional support to bereaved families. For example, when a disaster occurs, the AI ​​agent installed on the smartphone automatically activates. When the disaster victim records a message by voice, the AI ​​records and stores that message. Next, the AI ​​engages in dialogue with the disaster victim and provides support to alleviate anxiety. Furthermore, the stored messages are sent to family and friends after safety has been confirmed. This system provides a means for disaster victims to convey their last words and provides emotional support to bereaved families. In this way, the disaster message system can record and store the voice and thoughts of disaster victims, alleviate anxiety, and send messages, thereby providing a sense of security to disaster victims and their families and friends.

[0029] The disaster message system according to this embodiment comprises an activation unit, a recording unit, a dialogue unit, and a transmission unit. The activation unit is activated automatically in the event of a disaster. The activation unit can be activated automatically in the event of a disaster such as an earthquake, tsunami, or typhoon. The activation unit detects the occurrence of a disaster using sensors and activates the system. For example, the activation unit can be activated automatically when an earthquake occurs using a sensor that senses earthquake tremors. The activation unit can also be activated automatically when a tsunami occurs using a sensor that receives tsunami warnings. Furthermore, the activation unit can also be activated automatically when a typhoon occurs using a sensor that senses the approach of a typhoon. The recording unit records and stores the voices and thoughts of disaster victims. For example, the recording unit can record voice messages and store them as digital data. The recording unit can record and store not only voice messages but also text messages and images. For example, the recording unit records and stores text messages entered by disaster victims. The recording unit can also record and store images taken by disaster victims. The dialogue unit alleviates anxiety through dialogue based on the information recorded by the recording unit. The dialogue unit can, for example, conduct voice dialogues to alleviate the anxiety of disaster victims. The dialogue unit can also conduct text dialogues. For example, the dialogue unit can provide appropriate replies to text messages entered by disaster victims. The dialogue unit can also personalize the content of the dialogue. For example, the dialogue unit can adjust the content of the dialogue based on the disaster victim's past dialogue history. The transmission unit sends the saved messages. For example, the transmission unit can send saved voice messages to family and friends. The transmission unit can also send text messages and images. For example, the transmission unit can send saved text messages to family and friends. The transmission unit can also send saved images to family and friends. The transmission unit can also automatically select the recipient of the message. For example, the transmission unit can select an appropriate recipient based on the disaster victim's past contact history. As a result, the disaster message system according to this embodiment can record and store the voices and thoughts of disaster victims, alleviate their anxiety, and provide a sense of security to disaster victims and their families and friends by sending messages.

[0030] The activation unit automatically starts up in the event of a disaster. For example, it can automatically start up during disasters such as earthquakes, tsunamis, and typhoons. The activation unit uses sensors to detect the occurrence of a disaster and activates the system. For example, the activation unit can automatically start up when an earthquake occurs using sensors that detect seismic shaking. Specifically, it uses seismometers and acceleration sensors to detect the initial tremors (P-waves) and main tremors (S-waves) of an earthquake, and activates the system when a certain threshold is exceeded. The activation unit can also automatically start up when a tsunami occurs using sensors that receive tsunami warnings. Tsunami warnings are detected by receiving emergency warning signals (EWS) issued by the Japan Meteorological Agency and disaster prevention organizations. Furthermore, the activation unit can also automatically start up when a typhoon occurs using sensors that detect the approach of a typhoon. The approach of a typhoon is detected using barometric pressure sensors and anemometers, and the system activates when a specific drop in atmospheric pressure or increase in wind speed is confirmed. This allows the activation unit to quickly detect the occurrence of various disasters and automatically activate the system, enabling a rapid response to disaster victims. Furthermore, the activation unit can integrate data from multiple sensors to determine the type and scale of the disaster. For example, if both an earthquake and a tsunami occur, the activation unit can integrate this information to take a more appropriate response. This allows the activation unit to accurately detect the occurrence of a disaster and take a swift and appropriate response.

[0031] The recording unit records and stores the voices and thoughts of disaster victims. For example, the recording unit can record voice messages and store them as digital data. Specifically, voice messages are recorded when disaster victims speak into their smartphones or dedicated devices. The recorded audio is saved in digital format and stored on cloud servers or local storage. The recording unit can record and store not only voice messages, but also text messages and images. For example, text messages entered by disaster victims using their smartphone keyboards are recorded in real time and stored as digital data. Images taken by disaster victims using their smartphone cameras are also saved by the recording unit. This allows the recording unit to comprehensively record and store a variety of messages from disaster victims. Furthermore, the recording unit can encrypt the recorded data to ensure security. For example, voice messages, text messages, and image data are protected using encryption technologies such as AES (Advanced Encryption Standard) to prevent unauthorized access and data leakage. The recording unit also regularly backs up data to prevent data loss during disasters. In this way, the recording unit safely and securely stores important messages from disaster victims and makes them quickly accessible when needed.

[0032] The dialogue unit alleviates anxiety through dialogue based on information recorded by the recording unit. For example, the dialogue unit can conduct voice dialogues to alleviate the anxiety of disaster victims. Specifically, the dialogue unit uses natural language processing (NLP) technology to analyze the disaster victim's voice messages and generate appropriate responses. When a disaster victim speaks, the dialogue unit understands the content and provides emotional support by offering empathetic and encouraging words. The dialogue unit can also conduct text dialogues. For example, it can provide appropriate replies to text messages entered by disaster victims. The dialogue unit uses an AI chatbot to generate quick and appropriate responses to disaster victims' messages. Furthermore, the dialogue unit can personalize the content of the dialogue. For example, it can adjust the content of the dialogue based on the disaster victim's past dialogue history. By considering what the disaster victim has previously said and their emotional state, it can provide more individualized responses, thereby increasing the disaster victim's sense of security. In this way, the dialogue unit can provide emotional support to disaster victims and alleviate their anxiety. In addition, the dialogue unit can collect feedback from disaster victims and continuously improve the quality of the dialogues. For example, by having disaster victims evaluate the content of the dialogue, the dialogue department can use that evaluation to improve the accuracy and appropriateness of its responses. This allows the dialogue department to provide disaster victims with a better dialogue experience and better emotional support.

[0033] The transmission unit sends stored messages. For example, the transmission unit can send stored voice messages to family and friends. Specifically, the transmission unit processes voice messages recorded by the victim as digital data and sends them to designated recipients via the internet. The transmission unit can also send text messages and images. For example, text messages entered by the victim and images taken by the victim are converted to the appropriate format by the transmission unit and sent to family and friends. The transmission unit can also automatically select the recipient of the message. For example, the transmission unit selects the appropriate recipient based on the victim's past contact history. This allows the victim to send messages to those they want to contact quickly. Furthermore, the transmission unit can check the delivery status of sent messages and notify the victim whether the transmission was successful. For example, to confirm that the message has reached the recipient, the transmission unit receives a receipt confirmation notification and provides feedback to the victim. This allows the transmission unit to inform the victim that the message has been successfully sent and provide reassurance. The transmission unit can also send messages using multiple communication methods. For example, if the internet connection is unstable, messages can be sent using SMS or voice calls. This allows the transmission unit to reliably send messages in various situations, providing reassurance to disaster victims, their families, and friends.

[0034] The activation unit can be automatically activated in the event of a disaster such as a tsunami or landslide. For example, if a tsunami warning is issued, the activation unit will immediately activate and begin recording messages. The activation unit can also be activated after detecting earthquake tremors if there is a risk of a landslide. Furthermore, if a typhoon is expected to approach, the activation unit can issue a warning in advance and activate as needed. This allows for a rapid start of message recording by automatically activating during a disaster. Some or all of the above-described processes in the activation unit may be performed using AI, for example, or without AI. For example, the activation unit can input data on the received tsunami warning into a generating AI and have the generating AI execute the activation when a tsunami occurs.

[0035] The recording unit can record and store the voices and thoughts of disaster victims. For example, the recording unit can record voice messages from disaster victims and store them as digital data. The recording unit can also record and store text messages entered by disaster victims. Furthermore, the recording unit can record and store images taken by disaster victims. This allows the voices and thoughts of disaster victims to be recorded and stored, and later shared with family and friends. Some or all of the above processing in the recording unit may be performed using AI, for example, or without AI. For example, the recording unit can input voice data recorded by disaster victims into a generating AI and have the generating AI perform the storage of the voice data.

[0036] The dialogue unit can engage in conversations with disaster victims and provide support to alleviate their anxiety. For example, the dialogue unit can conduct voice conversations to alleviate the victims' anxiety. The dialogue unit can also conduct text conversations. For example, the dialogue unit can provide appropriate replies to text messages entered by disaster victims. Furthermore, the dialogue unit can personalize the content of the conversations. For example, the dialogue unit can adjust the content of the conversations based on the disaster victims' past conversation history. This allows the dialogue unit to engage in conversations with disaster victims and provide a sense of security by alleviating their anxiety. Some or all of the above processes in the dialogue unit may be performed using AI, for example, or not using AI. For example, the dialogue unit can input text messages entered by disaster victims into a generating AI and have the generating AI generate the content of the conversations.

[0037] The transmission unit can send saved messages to family and friends after safety has been confirmed. For example, the transmission unit can send saved voice messages to family and friends. The transmission unit can also send saved text messages and images. This allows the last words of the disaster victim to be conveyed by sending saved messages to family and friends. Some or all of the above processing in the transmission unit may be performed using AI, for example, or without AI. For example, the transmission unit can input saved messages into a generating AI and have the generating AI execute the message sending.

[0038] The activation unit can change its activation method depending on the type and scale of the disaster. For example, if a tsunami warning is issued, the activation unit will activate immediately and begin recording messages. The activation unit can also activate after detecting earthquake tremors if there is a risk of landslides. Furthermore, if a typhoon is expected to approach, the activation unit can issue a warning in advance and activate as needed. This allows for appropriate responses by changing the activation method according to the type and scale of the disaster. Some or all of the above processing in the activation unit may be performed using AI, for example, or without AI. For example, the activation unit can input data on the type and scale of the disaster into a generating AI and have the generating AI adjust the activation method.

[0039] The activation unit can more accurately detect the occurrence of a disaster by utilizing sensor information from a smartphone. For example, the activation unit can use the smartphone's accelerometer to detect earthquake tremors and activate. The activation unit can also use the smartphone's GPS information to predict the approach of a tsunami and activate. Furthermore, the activation unit can use the smartphone's barometric pressure sensor to detect the approach of a typhoon and activate. In this way, the occurrence of a disaster can be detected more accurately by utilizing the smartphone's sensor information. Some or all of the above processing in the activation unit may be performed using AI, for example, or without AI. For example, the activation unit can input the smartphone's sensor information into a generating AI and have the generating AI perform disaster detection.

[0040] The activation unit can be activated in conjunction with other devices (such as smartwatches and smart speakers). For example, the activation unit can use the heart rate sensor of a smartwatch to detect an anomaly and activate a smartphone. It can also use the voice recognition function of a smart speaker to detect the occurrence of a disaster and activate a smartphone. Furthermore, the activation unit can collaborate with other smart devices to share disaster information and activate a smartphone. This allows for more accurate detection of disasters and a faster response by collaborating with other devices. Some or all of the above-described processes in the activation unit may be performed using AI, for example, or without AI. For example, the activation unit can input heart rate data from a smartwatch into a generating AI and have the generating AI perform anomaly detection.

[0041] The activation unit can be activated only when a disaster occurs in a specific region, using geographical location information. For example, if the user is in a tsunami-prone area, the activation unit will activate when a tsunami warning is issued. The activation unit can also be activated after detecting earthquake tremors if the user is in a landslide-prone area. Furthermore, if the user is in the path of a typhoon, the activation unit can be activated when the typhoon is predicted to approach. This allows for appropriate responses by utilizing geographical location information to activate only when a disaster occurs in a specific region. Some or all of the above-described processes in the activation unit may be performed using AI, for example, or without AI. For example, the activation unit can input geographical location information into a generating AI and have the generating AI execute the activation in the event of a disaster.

[0042] The recording unit can record not only audio but also text and images. For example, the recording unit can allow users to input text messages when recording audio messages. It can also allow users to record disaster situations with images. Furthermore, the recording unit can allow users to record messages by combining audio, text, and images. This allows for the recording of a wider variety of messages by enabling the recording of text and images in addition to audio. Some or all of the above-described processes in the recording unit may be performed using AI, for example, or without AI. For example, the recording unit can input user-entered text and image data into a generating AI and have the generating AI record the data.

[0043] The recording unit can be equipped with a function to automatically classify and organize the content of recorded messages. For example, the recording unit can classify recorded messages based on the type of emotion (fear, relief, etc.). It can also organize recorded messages based on the importance of their content. Furthermore, the recording unit can classify recorded messages based on the recipient (family, friends, etc.). This makes it easier to find messages later by automatically classifying and organizing the content of recorded messages. Some or all of the above processing in the recording unit may be performed using AI, for example, or without AI. For example, the recording unit can input the data of recorded messages into a generating AI and have the generating AI perform the classification and organization.

[0044] The recording unit can back up recorded messages to the cloud in real time. For example, the recording unit can automatically back up messages to the cloud as soon as they are recorded. It can also back up messages to the cloud once recording is complete. Furthermore, the recording unit can periodically back up messages to the cloud as they are recorded. This improves data security by backing up recorded messages to the cloud in real time. Some or all of the above processes in the recording unit may be performed using AI, for example, or without AI. For example, the recording unit can input the data of recorded messages into a generating AI and have the generating AI perform the backup to the cloud.

[0045] The recording unit can encrypt and store recorded messages. For example, the recording unit can encrypt and store messages as soon as they are recorded. It can also encrypt and store messages after the recording is complete. Furthermore, the recording unit can periodically encrypt and store messages as they are recorded. This enhances data confidentiality by encrypting and storing recorded messages. Some or all of the above processes in the recording unit may be performed using AI, for example, or without AI. For example, the recording unit can input recorded message data into a generating AI and have the generating AI perform the process of encrypting and storing the data.

[0046] The dialogue unit can personalize the content of the dialogue based on the user's past dialogue history. For example, the dialogue unit can provide appropriate dialogue content based on the content of past conversations the user has had. The dialogue unit can also analyze the user's preferred dialogue style from their past dialogue history and personalize it accordingly. Furthermore, the dialogue unit can adjust the content of the dialogue by referring to the user's past dialogue history. This allows for more appropriate support to be provided by personalizing the content of the dialogue based on the user's past dialogue history. Some or all of the above processing in the dialogue unit may be performed using AI, for example, or without AI. For example, the dialogue unit can input the user's past dialogue history data into a generating AI and have the generating AI perform the personalization of the dialogue content.

[0047] The dialogue unit can provide the user with specific instructions and advice during the conversation. For example, if the user is feeling frightened, the dialogue unit can provide specific instructions to calm down. If the user is calm, the dialogue unit can also provide specific advice to help them understand the situation. Furthermore, if the user is in a state of panic, the dialogue unit can provide specific instructions to act quickly. In this way, by providing specific instructions and advice during the conversation, the user can take appropriate action. Some or all of the above processing in the dialogue unit may be performed using AI, for example, or without AI. For example, the dialogue unit can input user situation data into a generating AI and have the generating AI generate specific instructions and advice.

[0048] The dialogue unit can handle dialogue in multiple languages. For example, if the user speaks English, the dialogue unit will conduct the conversation in English. It can also conduct the conversation in Japanese if the user speaks Japanese. Furthermore, if the user speaks multiple languages, the dialogue unit can switch languages ​​during the conversation. This multilingual support allows the system to accommodate users who speak various languages. Some or all of the above processing in the dialogue unit may be performed using AI, or without AI. For example, the dialogue unit can input the user's language data into a generating AI and have the generating AI generate multilingual dialogue content.

[0049] The dialogue unit can provide the content of the dialogue not only in audio but also in text. For example, when a user is engaging in a dialogue via audio, the dialogue unit can also display the dialogue content in text. Furthermore, if the user cannot hear the audio, the dialogue unit can also provide the dialogue content in text. In addition, the dialogue unit can allow the user to confirm the dialogue content in both audio and text. This allows the user to confirm the dialogue content according to the situation by providing the dialogue content in both audio and text. Some or all of the above processing in the dialogue unit may be performed using AI, for example, or without AI. For example, the dialogue unit can input audio data into a generating AI and have the generating AI perform the conversion to text data.

[0050] The sending unit can automatically select message recipients based on the user's past contact history. For example, the sending unit can prioritize recipients who the user has frequently contacted in the past. The sending unit can also automatically select family and close friends from the user's contact history. Furthermore, the sending unit can analyze the user's past contact history and suggest the most appropriate recipients. This allows messages to be sent to the right people by automatically selecting recipients based on the user's past contact history. Some or all of the above processing in the sending unit may be performed using AI, for example, or not. For example, the sending unit can input the user's contact history data into a generating AI and have the generating AI perform the recipient selection.

[0051] The sending unit can automatically check the content of a message and correct inappropriate expressions before sending it. For example, the sending unit can use AI to automatically check the content of the message and correct inappropriate expressions. Alternatively, the sending unit can have AI review the content of the message before sending and correct it to appropriate expressions. Furthermore, the sending unit can have AI analyze the content of the message and correct inappropriate expressions before sending it. This allows for the sending of appropriate messages by checking the content and correcting inappropriate expressions before sending. Some or all of the above-described processes in the sending unit may be performed using AI, for example, or without AI. For example, the sending unit can input message content data into a generating AI and have the generating AI correct inappropriate expressions.

[0052] The sending unit can send messages to multiple platforms (email, social networking services, messaging apps, etc.). For example, the sending unit can send messages to multiple platforms such as email, social networking services, and messaging apps. The sending unit can also automatically select message recipients based on user settings. Furthermore, by sending messages to multiple platforms, the sending unit can ensure that messages are delivered reliably. This ensures that messages are delivered reliably by supporting multiple platform recipients. Some or all of the above processing in the sending unit may be performed using AI, for example, or without AI. For example, the sending unit can input message recipient data into a generating AI and have the generating AI execute the sending to multiple platforms.

[0053] The sending unit can be configured to include a function to check the recipient's reception status after the message has been sent. For example, after the message has been sent, the AI ​​can automatically check the recipient's reception status. The sending unit can also check the message reception status in real time and notify whether the recipient has received the message. Furthermore, the sending unit can periodically check the recipient's reception status after the message has been sent. This allows for confirmation that the message was delivered reliably by checking the reception status after sending. Some or all of the above processing in the sending unit may be performed using AI, or not using AI. For example, the sending unit can input the recipient's reception status data into a generating AI and have the generating AI perform the reception status check.

[0054] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0055] The activation unit not only automatically activates during disasters, but can also activate if it monitors the user's health and detects an abnormality. For example, the activation unit can use the heart rate sensor of a smartwatch to activate if the user's heart rate is abnormally high. It can also use the accelerometer sensor of a smartphone to activate if the user falls. Furthermore, the activation unit can monitor the user's respiratory rate and activate if it detects an abnormality. This allows the system to be activated at the appropriate time according to the user's health condition, not just during disasters.

[0056] The recording unit can not only record the voices and thoughts of disaster victims, but also record their location information. For example, when a disaster victim records a message, the recording unit acquires GPS information and records their location. The recording unit can also periodically record the location information of disaster victims as they move. Furthermore, the recording unit can record the location information of disaster victims when they arrive at an evacuation center. By recording the location information of disaster victims, family and friends can later determine their whereabouts.

[0057] The dialogue unit not only engages in dialogue with disaster victims but also monitors their health and contacts medical institutions as needed. For example, the dialogue unit monitors the victim's heart rate and respiratory rate and contacts medical institutions if it detects any abnormalities. It can also contact medical institutions if a victim complains of feeling unwell. Furthermore, it can contact medical institutions if a victim is injured. This allows for a swift response by monitoring the victim's health and contacting medical institutions as needed.

[0058] The sending unit not only sends stored messages, but also checks the recipient's reception status and can resend messages if they are not delivered. For example, after a message is sent, the sending unit checks the recipient's reception status and resends the message if it is not delivered. The sending unit can also check the message reception status in real time and notify whether the recipient has received the message. Furthermore, after sending a message, the sending unit can periodically check the recipient's reception status and resend the message as needed. This ensures that messages are delivered reliably.

[0059] The following briefly describes the processing flow for example form 1.

[0060] Step 1: The activation unit automatically starts up in the event of a disaster. The activation unit uses sensors to detect the occurrence of disasters such as earthquakes, tsunamis, and typhoons, and activates the system. For example, by using sensors that detect earthquake tremors, sensors that receive tsunami warnings, and sensors that detect approaching typhoons, the system can be automatically activated when each type of disaster occurs. Step 2: The recording unit records and stores the voices and thoughts of the disaster victims. In addition to recording voice messages and storing them as digital data, the recording unit can also record and store text messages and images. For example, it records and stores text messages entered by disaster victims and images taken by them. Step 3: The dialogue unit alleviates anxiety through dialogue based on the information recorded by the recording unit. The dialogue unit can alleviate the anxiety of disaster victims through voice or text dialogue. The dialogue unit can also personalize the content of the dialogue based on the disaster victim's past dialogue history. Step 4: The sender sends the saved messages. The sender can send saved voice messages, text messages, and images to family and friends. The sender can also automatically select appropriate recipients based on the victim's past contact history.

[0061] (Example of form 2) The disaster message system according to an embodiment of the present invention is a system that delivers a final message to a loved one when facing a life-threatening situation during a disaster. This system is installed on a smartphone and automatically activates during disasters such as tsunamis and landslides, recording and storing the voice and thoughts of the disaster victim. The system uses AI to alleviate anxiety through dialogue and provides a sense of security by supporting those who are feeling lonely. Furthermore, the stored messages are sent to family and friends after safety has been confirmed. This system allows disaster victims to convey their last words and provides emotional support to bereaved families. For example, when a disaster occurs, the AI ​​agent installed on the smartphone automatically activates. When the disaster victim records a message by voice, the AI ​​records and stores that message. Next, the AI ​​engages in dialogue with the disaster victim and provides support to alleviate anxiety. Furthermore, the stored messages are sent to family and friends after safety has been confirmed. This system provides a means for disaster victims to convey their last words and provides emotional support to bereaved families. In this way, the disaster message system can record and store the voice and thoughts of disaster victims, alleviate anxiety, and send messages, thereby providing a sense of security to disaster victims and their families and friends.

[0062] The disaster message system according to this embodiment comprises an activation unit, a recording unit, a dialogue unit, and a transmission unit. The activation unit is activated automatically in the event of a disaster. The activation unit can be activated automatically in the event of a disaster such as an earthquake, tsunami, or typhoon. The activation unit detects the occurrence of a disaster using sensors and activates the system. For example, the activation unit can be activated automatically when an earthquake occurs using a sensor that senses earthquake tremors. The activation unit can also be activated automatically when a tsunami occurs using a sensor that receives tsunami warnings. Furthermore, the activation unit can also be activated automatically when a typhoon occurs using a sensor that senses the approach of a typhoon. The recording unit records and stores the voices and thoughts of disaster victims. For example, the recording unit can record voice messages and store them as digital data. The recording unit can record and store not only voice messages but also text messages and images. For example, the recording unit records and stores text messages entered by disaster victims. The recording unit can also record and store images taken by disaster victims. The dialogue unit alleviates anxiety through dialogue based on the information recorded by the recording unit. The dialogue unit can, for example, conduct voice dialogues to alleviate the anxiety of disaster victims. The dialogue unit can also conduct text dialogues. For example, the dialogue unit can provide appropriate replies to text messages entered by disaster victims. The dialogue unit can also personalize the content of the dialogue. For example, the dialogue unit can adjust the content of the dialogue based on the disaster victim's past dialogue history. The transmission unit sends the saved messages. For example, the transmission unit can send saved voice messages to family and friends. The transmission unit can also send text messages and images. For example, the transmission unit can send saved text messages to family and friends. The transmission unit can also send saved images to family and friends. The transmission unit can also automatically select the recipient of the message. For example, the transmission unit can select an appropriate recipient based on the disaster victim's past contact history. As a result, the disaster message system according to this embodiment can record and store the voices and thoughts of disaster victims, alleviate their anxiety, and provide a sense of security to disaster victims and their families and friends by sending messages.

[0063] The activation unit automatically starts up in the event of a disaster. For example, it can automatically start up during disasters such as earthquakes, tsunamis, and typhoons. The activation unit uses sensors to detect the occurrence of a disaster and activates the system. For example, the activation unit can automatically start up when an earthquake occurs using sensors that detect seismic shaking. Specifically, it uses seismometers and acceleration sensors to detect the initial tremors (P-waves) and main tremors (S-waves) of an earthquake, and activates the system when a certain threshold is exceeded. The activation unit can also automatically start up when a tsunami occurs using sensors that receive tsunami warnings. Tsunami warnings are detected by receiving emergency warning signals (EWS) issued by the Japan Meteorological Agency and disaster prevention organizations. Furthermore, the activation unit can also automatically start up when a typhoon occurs using sensors that detect the approach of a typhoon. The approach of a typhoon is detected using barometric pressure sensors and anemometers, and the system activates when a specific drop in atmospheric pressure or increase in wind speed is confirmed. This allows the activation unit to quickly detect the occurrence of various disasters and automatically activate the system, enabling a rapid response to disaster victims. Furthermore, the activation unit can integrate data from multiple sensors to determine the type and scale of the disaster. For example, if both an earthquake and a tsunami occur, the activation unit can integrate this information to take a more appropriate response. This allows the activation unit to accurately detect the occurrence of a disaster and take a swift and appropriate response.

[0064] The recording unit records and stores the voices and thoughts of disaster victims. For example, the recording unit can record voice messages and store them as digital data. Specifically, voice messages are recorded when disaster victims speak into their smartphones or dedicated devices. The recorded audio is saved in digital format and stored on cloud servers or local storage. The recording unit can record and store not only voice messages, but also text messages and images. For example, text messages entered by disaster victims using their smartphone keyboards are recorded in real time and stored as digital data. Images taken by disaster victims using their smartphone cameras are also saved by the recording unit. This allows the recording unit to comprehensively record and store a variety of messages from disaster victims. Furthermore, the recording unit can encrypt the recorded data to ensure security. For example, voice messages, text messages, and image data are protected using encryption technologies such as AES (Advanced Encryption Standard) to prevent unauthorized access and data leakage. The recording unit also regularly backs up data to prevent data loss during disasters. In this way, the recording unit safely and securely stores important messages from disaster victims and makes them quickly accessible when needed.

[0065] The dialogue unit alleviates anxiety through dialogue based on information recorded by the recording unit. For example, the dialogue unit can conduct voice dialogues to alleviate the anxiety of disaster victims. Specifically, the dialogue unit uses natural language processing (NLP) technology to analyze the disaster victim's voice messages and generate appropriate responses. When a disaster victim speaks, the dialogue unit understands the content and provides emotional support by offering empathetic and encouraging words. The dialogue unit can also conduct text dialogues. For example, it can provide appropriate replies to text messages entered by disaster victims. The dialogue unit uses an AI chatbot to generate quick and appropriate responses to disaster victims' messages. Furthermore, the dialogue unit can personalize the content of the dialogue. For example, it can adjust the content of the dialogue based on the disaster victim's past dialogue history. By considering what the disaster victim has previously said and their emotional state, it can provide more individualized responses, thereby increasing the disaster victim's sense of security. In this way, the dialogue unit can provide emotional support to disaster victims and alleviate their anxiety. In addition, the dialogue unit can collect feedback from disaster victims and continuously improve the quality of the dialogues. For example, by having disaster victims evaluate the content of the dialogue, the dialogue department can use that evaluation to improve the accuracy and appropriateness of its responses. This allows the dialogue department to provide disaster victims with a better dialogue experience and better emotional support.

[0066] The transmission unit sends stored messages. For example, the transmission unit can send stored voice messages to family and friends. Specifically, the transmission unit processes voice messages recorded by the victim as digital data and sends them to designated recipients via the internet. The transmission unit can also send text messages and images. For example, text messages entered by the victim and images taken by the victim are converted to the appropriate format by the transmission unit and sent to family and friends. The transmission unit can also automatically select the recipient of the message. For example, the transmission unit selects the appropriate recipient based on the victim's past contact history. This allows the victim to send messages to those they want to contact quickly. Furthermore, the transmission unit can check the delivery status of sent messages and notify the victim whether the transmission was successful. For example, to confirm that the message has reached the recipient, the transmission unit receives a receipt confirmation notification and provides feedback to the victim. This allows the transmission unit to inform the victim that the message has been successfully sent and provide reassurance. The transmission unit can also send messages using multiple communication methods. For example, if the internet connection is unstable, messages can be sent using SMS or voice calls. This allows the transmission unit to reliably send messages in various situations, providing reassurance to disaster victims, their families, and friends.

[0067] The activation unit can be automatically activated in the event of a disaster such as a tsunami or landslide. For example, if a tsunami warning is issued, the activation unit will immediately activate and begin recording messages. The activation unit can also be activated after detecting earthquake tremors if there is a risk of a landslide. Furthermore, if a typhoon is expected to approach, the activation unit can issue a warning in advance and activate as needed. This allows for a rapid start of message recording by automatically activating during a disaster. Some or all of the above-described processes in the activation unit may be performed using AI, for example, or without AI. For example, the activation unit can input data on the received tsunami warning into a generating AI and have the generating AI execute the activation when a tsunami occurs.

[0068] The recording unit can record and store the voices and thoughts of disaster victims. For example, the recording unit can record voice messages from disaster victims and store them as digital data. The recording unit can also record and store text messages entered by disaster victims. Furthermore, the recording unit can record and store images taken by disaster victims. This allows the voices and thoughts of disaster victims to be recorded and stored, and later shared with family and friends. Some or all of the above processing in the recording unit may be performed using AI, for example, or without AI. For example, the recording unit can input voice data recorded by disaster victims into a generating AI and have the generating AI perform the storage of the voice data.

[0069] The dialogue unit can engage in conversations with disaster victims and provide support to alleviate their anxiety. For example, the dialogue unit can conduct voice conversations to alleviate the victims' anxiety. The dialogue unit can also conduct text conversations. For example, the dialogue unit can provide appropriate replies to text messages entered by disaster victims. Furthermore, the dialogue unit can personalize the content of the conversations. For example, the dialogue unit can adjust the content of the conversations based on the disaster victims' past conversation history. This allows the dialogue unit to engage in conversations with disaster victims and provide a sense of security by alleviating their anxiety. Some or all of the above processes in the dialogue unit may be performed using AI, for example, or not using AI. For example, the dialogue unit can input text messages entered by disaster victims into a generating AI and have the generating AI generate the content of the conversations.

[0070] The transmission unit can send saved messages to family and friends after safety has been confirmed. For example, the transmission unit can send saved voice messages to family and friends. The transmission unit can also send saved text messages and images. This allows the last words of the disaster victim to be conveyed by sending saved messages to family and friends. Some or all of the above processing in the transmission unit may be performed using AI, for example, or without AI. For example, the transmission unit can input saved messages into a generating AI and have the generating AI execute the message sending.

[0071] The activation unit can estimate the user's emotions and adjust the activation timing based on the estimated emotions. For example, if the user is experiencing strong fear, the activation unit can activate immediately and begin recording a message. Alternatively, if the user is calm, the activation unit can assess the situation before activating. Furthermore, if the user is in a state of panic, the activation unit can quickly activate and begin a conversation to provide reassurance. By adjusting the activation timing according to the user's emotions, message recording can be started at a more appropriate time. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processes in the activation unit may be performed using AI or not. For example, the activation unit can input user emotion data into the generative AI and have the generative AI adjust the activation timing.

[0072] The activation unit can change its activation method depending on the type and scale of the disaster. For example, if a tsunami warning is issued, the activation unit will activate immediately and begin recording messages. The activation unit can also activate after detecting earthquake tremors if there is a risk of landslides. Furthermore, if a typhoon is expected to approach, the activation unit can issue a warning in advance and activate as needed. This allows for appropriate responses by changing the activation method according to the type and scale of the disaster. Some or all of the above processing in the activation unit may be performed using AI, for example, or without AI. For example, the activation unit can input data on the type and scale of the disaster into a generating AI and have the generating AI adjust the activation method.

[0073] The activation unit can more accurately detect the occurrence of a disaster by utilizing sensor information from a smartphone. For example, the activation unit can use the smartphone's accelerometer to detect earthquake tremors and activate. The activation unit can also use the smartphone's GPS information to predict the approach of a tsunami and activate. Furthermore, the activation unit can use the smartphone's barometric pressure sensor to detect the approach of a typhoon and activate. In this way, the occurrence of a disaster can be detected more accurately by utilizing the smartphone's sensor information. Some or all of the above processing in the activation unit may be performed using AI, for example, or without AI. For example, the activation unit can input the smartphone's sensor information into a generating AI and have the generating AI perform disaster detection.

[0074] The activation unit can estimate the user's emotions and determine activation priorities based on the estimated emotions. For example, if the user is experiencing strong fear, the activation unit will activate with the highest priority. Alternatively, if the user is calm, the activation unit may prioritize other important tasks before activating. Furthermore, if the user is in a state of panic, the activation unit can quickly activate and initiate a conversation to provide reassurance. This allows for message recording to begin at a more appropriate time by determining activation priorities according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processes in the activation unit may be performed using AI or not. For example, the activation unit can input user emotion data into a generative AI and have the generative AI determine the activation priorities.

[0075] The activation unit can be activated in conjunction with other devices (such as smartwatches and smart speakers). For example, the activation unit can use the heart rate sensor of a smartwatch to detect an anomaly and activate a smartphone. It can also use the voice recognition function of a smart speaker to detect the occurrence of a disaster and activate a smartphone. Furthermore, the activation unit can collaborate with other smart devices to share disaster information and activate a smartphone. This allows for more accurate detection of disasters and a faster response by collaborating with other devices. Some or all of the above-described processes in the activation unit may be performed using AI, for example, or without AI. For example, the activation unit can input heart rate data from a smartwatch into a generating AI and have the generating AI perform anomaly detection.

[0076] The activation unit can be activated only when a disaster occurs in a specific region, using geographical location information. For example, if the user is in a tsunami-prone area, the activation unit will activate when a tsunami warning is issued. The activation unit can also be activated after detecting earthquake tremors if the user is in a landslide-prone area. Furthermore, if the user is in the path of a typhoon, the activation unit can be activated when the typhoon is predicted to approach. This allows for appropriate responses by utilizing geographical location information to activate only when a disaster occurs in a specific region. Some or all of the above-described processes in the activation unit may be performed using AI, for example, or without AI. For example, the activation unit can input geographical location information into a generating AI and have the generating AI execute the activation in the event of a disaster.

[0077] The recording unit can estimate the user's emotions and adjust the recording method based on the estimated emotions. For example, if the user is feeling intense fear, the recording unit may prompt them to record a concise message. If the user is calm, the recording unit may also prompt them to record a detailed message. Furthermore, if the user is in a state of panic, the recording unit may prompt them to record a message that provides reassurance. This allows for the recording of more appropriate messages by adjusting the recording method according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the recording unit may be performed using AI or not. For example, the recording unit can input user emotion data into a generative AI and have the generative AI adjust the recording method.

[0078] The recording unit can record not only audio but also text and images. For example, the recording unit can allow users to input text messages when recording audio messages. It can also allow users to record disaster situations with images. Furthermore, the recording unit can allow users to record messages by combining audio, text, and images. This allows for the recording of a wider variety of messages by enabling the recording of text and images in addition to audio. Some or all of the above-described processes in the recording unit may be performed using AI, for example, or without AI. For example, the recording unit can input user-entered text and image data into a generating AI and have the generating AI record the data.

[0079] The recording unit can be equipped with a function to automatically classify and organize the content of recorded messages. For example, the recording unit can classify recorded messages based on the type of emotion (fear, relief, etc.). It can also organize recorded messages based on the importance of their content. Furthermore, the recording unit can classify recorded messages based on the recipient (family, friends, etc.). This makes it easier to find messages later by automatically classifying and organizing the content of recorded messages. Some or all of the above processing in the recording unit may be performed using AI, for example, or without AI. For example, the recording unit can input the data of recorded messages into a generating AI and have the generating AI perform the classification and organization.

[0080] The recording unit can estimate the user's emotions and determine recording priorities based on the estimated emotions. For example, if the user is experiencing strong fear, the recording unit will record the message with the highest priority. If the user is calm, the recording unit can prioritize other important tasks before recording the message. Furthermore, if the user is in a state of panic, the recording unit can record the message quickly. This allows for recording messages at a more appropriate time by determining recording priorities according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the recording unit may be performed using AI or not. For example, the recording unit can input user emotion data into a generative AI and have the generative AI determine the recording priorities.

[0081] The recording unit can back up recorded messages to the cloud in real time. For example, the recording unit can automatically back up messages to the cloud as soon as they are recorded. It can also back up messages to the cloud once recording is complete. Furthermore, the recording unit can periodically back up messages to the cloud as they are recorded. This improves data security by backing up recorded messages to the cloud in real time. Some or all of the above processes in the recording unit may be performed using AI, for example, or without AI. For example, the recording unit can input the data of recorded messages into a generating AI and have the generating AI perform the backup to the cloud.

[0082] The recording unit can encrypt and store recorded messages. For example, the recording unit can encrypt and store messages as soon as they are recorded. It can also encrypt and store messages after the recording is complete. Furthermore, the recording unit can periodically encrypt and store messages as they are recorded. This enhances data confidentiality by encrypting and storing recorded messages. Some or all of the above processes in the recording unit may be performed using AI, for example, or without AI. For example, the recording unit can input recorded message data into a generating AI and have the generating AI perform the process of encrypting and storing the data.

[0083] The dialogue unit can estimate the user's emotions and adjust the content of the dialogue based on the estimated emotions. For example, if the user is experiencing strong fear, the dialogue unit can engage in dialogue to provide reassurance. If the user is calm, the dialogue unit can also engage in dialogue to help them understand the situation. Furthermore, if the user is in a state of panic, the dialogue unit can engage in dialogue to calm them down. By adjusting the content of the dialogue according to the user's emotions, more appropriate support can be provided. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the dialogue unit may be performed using AI, or not using AI. For example, the dialogue unit can input user emotion data into a generative AI and have the generative AI adjust the content of the dialogue.

[0084] The dialogue unit can personalize the content of the dialogue based on the user's past dialogue history. For example, the dialogue unit can provide appropriate dialogue content based on the content of past conversations the user has had. The dialogue unit can also analyze the user's preferred dialogue style from their past dialogue history and personalize it accordingly. Furthermore, the dialogue unit can adjust the content of the dialogue by referring to the user's past dialogue history. This allows for more appropriate support to be provided by personalizing the content of the dialogue based on the user's past dialogue history. Some or all of the above processing in the dialogue unit may be performed using AI, for example, or without AI. For example, the dialogue unit can input the user's past dialogue history data into a generating AI and have the generating AI perform the personalization of the dialogue content.

[0085] The dialogue unit can provide the user with specific instructions and advice during the conversation. For example, if the user is feeling frightened, the dialogue unit can provide specific instructions to calm down. If the user is calm, the dialogue unit can also provide specific advice to help them understand the situation. Furthermore, if the user is in a state of panic, the dialogue unit can provide specific instructions to act quickly. In this way, by providing specific instructions and advice during the conversation, the user can take appropriate action. Some or all of the above processing in the dialogue unit may be performed using AI, for example, or without AI. For example, the dialogue unit can input user situation data into a generating AI and have the generating AI generate specific instructions and advice.

[0086] The dialogue unit can estimate the user's emotions and determine the priority of the dialogue based on the estimated emotions. For example, if the user is experiencing strong fear, the dialogue unit will prioritize the conversation. If the user is calm, the dialogue unit can also prioritize other important tasks before engaging in dialogue. Furthermore, if the user is in a state of panic, the dialogue unit can engage in conversation quickly. This allows for more timely support by prioritizing dialogue according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the dialogue unit may be performed using AI or not. For example, the dialogue unit can input user emotion data into a generative AI and have the generative AI determine the priority of the dialogue.

[0087] The dialogue unit can handle dialogue in multiple languages. For example, if the user speaks English, the dialogue unit will conduct the conversation in English. It can also conduct the conversation in Japanese if the user speaks Japanese. Furthermore, if the user speaks multiple languages, the dialogue unit can switch languages ​​during the conversation. This multilingual support allows the system to accommodate users who speak various languages. Some or all of the above processing in the dialogue unit may be performed using AI, or without AI. For example, the dialogue unit can input the user's language data into a generating AI and have the generating AI generate multilingual dialogue content.

[0088] The dialogue unit can provide the content of the dialogue not only in audio but also in text. For example, when a user is engaging in a dialogue via audio, the dialogue unit can also display the dialogue content in text. Furthermore, if the user cannot hear the audio, the dialogue unit can also provide the dialogue content in text. In addition, the dialogue unit can allow the user to confirm the dialogue content in both audio and text. This allows the user to confirm the dialogue content according to the situation by providing the dialogue content in both audio and text. Some or all of the above processing in the dialogue unit may be performed using AI, for example, or without AI. For example, the dialogue unit can input audio data into a generating AI and have the generating AI perform the conversion to text data.

[0089] The sending unit can estimate the user's emotions and adjust the timing of transmission based on the estimated emotions. For example, if the user is experiencing strong fear, the sending unit can immediately send a message. Alternatively, if the user is calm, the sending unit can assess the situation before sending a message. Furthermore, if the user is in a state of panic, the sending unit can send a message quickly. By adjusting the timing of transmission according to the user's emotions, messages can be sent at a more appropriate time. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processes in the sending unit may be performed using AI or not. For example, the sending unit can input user emotion data into a generative AI and have the generative AI adjust the timing of transmission.

[0090] The sending unit can automatically select message recipients based on the user's past contact history. For example, the sending unit can prioritize recipients who the user has frequently contacted in the past. The sending unit can also automatically select family and close friends from the user's contact history. Furthermore, the sending unit can analyze the user's past contact history and suggest the most appropriate recipients. This allows messages to be sent to the right people by automatically selecting recipients based on the user's past contact history. Some or all of the above processing in the sending unit may be performed using AI, for example, or not. For example, the sending unit can input the user's contact history data into a generating AI and have the generating AI perform the recipient selection.

[0091] The sending unit can automatically check the content of a message and correct inappropriate expressions before sending it. For example, the sending unit can use AI to automatically check the content of the message and correct inappropriate expressions. Alternatively, the sending unit can have AI review the content of the message before sending and correct it to appropriate expressions. Furthermore, the sending unit can have AI analyze the content of the message and correct inappropriate expressions before sending it. This allows for the sending of appropriate messages by checking the content and correcting inappropriate expressions before sending. Some or all of the above-described processes in the sending unit may be performed using AI, for example, or without AI. For example, the sending unit can input message content data into a generating AI and have the generating AI correct inappropriate expressions.

[0092] The sending unit can estimate the user's emotions and determine the priority of sending messages based on the estimated emotions. For example, if the user is experiencing strong fear, the sending unit will send the message with the highest priority. If the user is calm, the sending unit can also prioritize other important tasks before sending the message. Furthermore, if the user is in a state of panic, the sending unit can send the message quickly. This allows for more appropriate timing of message delivery by prioritizing messages according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processes in the sending unit may be performed using AI or not. For example, the sending unit can input user emotion data into a generative AI and have the generative AI determine the priority of sending messages.

[0093] The sending unit can send messages to multiple platforms (email, social networking services, messaging apps, etc.). For example, the sending unit can send messages to multiple platforms such as email, social networking services, and messaging apps. The sending unit can also automatically select message recipients based on user settings. Furthermore, by sending messages to multiple platforms, the sending unit can ensure that messages are delivered reliably. This ensures that messages are delivered reliably by supporting multiple platform recipients. Some or all of the above processing in the sending unit may be performed using AI, for example, or without AI. For example, the sending unit can input message recipient data into a generating AI and have the generating AI execute the sending to multiple platforms.

[0094] The sending unit can be configured to include a function to check the recipient's reception status after the message has been sent. For example, after the message has been sent, the AI ​​can automatically check the recipient's reception status. The sending unit can also check the message reception status in real time and notify whether the recipient has received the message. Furthermore, the sending unit can periodically check the recipient's reception status after the message has been sent. This allows for confirmation that the message was delivered reliably by checking the reception status after sending. Some or all of the above processing in the sending unit may be performed using AI, or not using AI. For example, the sending unit can input the recipient's reception status data into a generating AI and have the generating AI perform the reception status check.

[0095] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0096] The activation unit not only automatically activates during disasters, but can also activate if it monitors the user's health and detects an abnormality. For example, the activation unit can use the heart rate sensor of a smartwatch to activate if the user's heart rate is abnormally high. It can also use the accelerometer sensor of a smartphone to activate if the user falls. Furthermore, the activation unit can monitor the user's respiratory rate and activate if it detects an abnormality. This allows the system to be activated at the appropriate time according to the user's health condition, not just during disasters.

[0097] The recording unit can not only record the voices and thoughts of disaster victims, but also record their location information. For example, when a disaster victim records a message, the recording unit acquires GPS information and records their location. The recording unit can also periodically record the location information of disaster victims as they move. Furthermore, the recording unit can record the location information of disaster victims when they arrive at an evacuation center. By recording the location information of disaster victims, family and friends can later determine their whereabouts.

[0098] The dialogue unit not only engages in dialogue with disaster victims but also monitors their health and contacts medical institutions as needed. For example, the dialogue unit monitors the victim's heart rate and respiratory rate and contacts medical institutions if it detects any abnormalities. It can also contact medical institutions if a victim complains of feeling unwell. Furthermore, it can contact medical institutions if a victim is injured. This allows for a swift response by monitoring the victim's health and contacting medical institutions as needed.

[0099] The sending unit not only sends stored messages, but also checks the recipient's reception status and can resend messages if they are not delivered. For example, after a message is sent, the sending unit checks the recipient's reception status and resends the message if it is not delivered. The sending unit can also check the message reception status in real time and notify whether the recipient has received the message. Furthermore, after sending a message, the sending unit can periodically check the recipient's reception status and resend the message as needed. This ensures that messages are delivered reliably.

[0100] The activation unit not only automatically activates during a disaster, but can also estimate the user's emotions and adjust the activation timing based on those emotions. For example, if the user is experiencing intense fear, the activation unit will activate immediately and begin recording messages. Alternatively, if the user is calm, the activation unit can assess the situation before activating. Furthermore, if the user is in a state of panic, the activation unit can quickly activate and initiate a conversation to provide reassurance. This allows for adjusting the activation timing according to the user's emotions, enabling message recording to begin at a more appropriate time.

[0101] The recording unit not only records the voices and thoughts of disaster victims, but can also estimate their emotions and adjust the content of the message based on those estimated emotions. For example, if a disaster victim is experiencing intense fear, the recording unit may encourage them to record a concise message. If the disaster victim is calm, the recording unit may encourage them to record a more detailed message. Furthermore, if the disaster victim is in a state of panic, the recording unit may encourage them to record a message that provides reassurance. By adjusting the content of the message according to the disaster victim's emotions, a more appropriate message can be recorded.

[0102] The dialogue unit not only engages in dialogue with disaster victims, but can also estimate their emotions and adjust the content of the dialogue based on those estimates. For example, if a disaster victim is experiencing intense fear, the dialogue unit can engage in dialogue to provide reassurance. If the disaster victim is calm, the dialogue unit can also engage in dialogue to help them understand the situation. Furthermore, if the disaster victim is in a state of panic, the dialogue unit can engage in dialogue to help them calm down. By adjusting the content of the dialogue according to the disaster victim's emotions, more appropriate support can be provided.

[0103] The sending unit not only sends stored messages, but can also estimate the recipient's emotions and adjust the timing of the message based on those emotions. For example, if the recipient is experiencing intense fear, the sending unit will send the message immediately. Alternatively, if the recipient is calm, the sending unit can assess the situation before sending the message. Furthermore, if the recipient is in a state of panic, the sending unit can send the message quickly. This allows for more appropriate timing of the message delivery by adjusting the sending timing according to the recipient's emotions.

[0104] The sending unit can not only send stored messages, but also estimate the recipient's emotions and determine the sending priority based on those emotions. For example, if the recipient is experiencing intense fear, the sending unit will send the message with the highest priority. If the recipient is calm, the sending unit can also prioritize other important tasks before sending the message. Furthermore, if the recipient is in a state of panic, the sending unit can send the message quickly. This allows for more timely message delivery by prioritizing messages based on the recipient's emotions.

[0105] The sending unit can not only send stored messages, but also estimate the recipient's emotions and adjust the message content based on those emotions. For example, if the recipient is experiencing intense fear, the sending unit can send a message to provide reassurance. If the recipient is calm, the sending unit can also send a message to help them understand the situation. Furthermore, if the recipient is in a state of panic, the sending unit can send a message to calm them down. This allows for the sending of more appropriate messages by adjusting the content according to the recipient's emotions.

[0106] The following briefly describes the processing flow for example form 2.

[0107] Step 1: The activation unit automatically starts up in the event of a disaster. The activation unit uses sensors to detect the occurrence of disasters such as earthquakes, tsunamis, and typhoons, and activates the system. For example, by using sensors that detect earthquake tremors, sensors that receive tsunami warnings, and sensors that detect approaching typhoons, the system can be automatically activated when each type of disaster occurs. Step 2: The recording unit records and stores the voices and thoughts of the disaster victims. In addition to recording voice messages and storing them as digital data, the recording unit can also record and store text messages and images. For example, it records and stores text messages entered by disaster victims and images taken by them. Step 3: The dialogue unit alleviates anxiety through dialogue based on the information recorded by the recording unit. The dialogue unit can alleviate the anxiety of disaster victims through voice or text dialogue. The dialogue unit can also personalize the content of the dialogue based on the disaster victim's past dialogue history. Step 4: The sender sends the saved messages. The sender can send saved voice messages, text messages, and images to family and friends. The sender can also automatically select appropriate recipients based on the victim's past contact history.

[0108] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0109] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0110] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0111] Each of the multiple elements described above, including the activation unit, recording unit, dialogue unit, and transmission unit, is implemented in at least one of the smart device 14 and the data processing unit 12. For example, the activation unit detects the occurrence of a disaster using the sensors of the smart device 14 and automatically activates the system. The recording unit records the voices of disaster victims using the microphone 38B of the smart device 14 and stores them as digital data. The dialogue unit conducts voice dialogue using the control unit 46A of the smart device 14 to alleviate the anxiety of disaster victims. The transmission unit sends the stored messages to family and friends using the specific processing unit 290 of the data processing unit 12. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0112] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0113] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0114] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0115] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0116] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0117] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0118] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0119] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0120] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0121] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0122] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0123] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0124] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0125] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0126] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0127] Each of the multiple elements described above, including the activation unit, recording unit, dialogue unit, and transmission unit, is implemented, for example, in at least one of the smart glasses 214 and the data processing unit 12. For example, the activation unit detects the occurrence of a disaster using the sensors of the smart glasses 214 and automatically activates the system. The recording unit records the voice of the disaster victim using the microphone 238 of the smart glasses 214 and stores it as digital data. The dialogue unit conducts voice dialogue using the control unit 46A of the smart glasses 214 to alleviate the anxiety of the disaster victim. The transmission unit sends the stored message to family and friends using the specific processing unit 290 of the data processing unit 12. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0128] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0129] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0130] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0131] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0132] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0133] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0134] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0135] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0136] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0137] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0138] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0139] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0140] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0141] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0142] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0143] Each of the multiple elements described above, including the startup unit, recording unit, dialogue unit, and transmission unit, is implemented in, for example, at least one of the headset terminal 314 and the data processing unit 12. For example, the startup unit detects the occurrence of a disaster using the sensors of the headset terminal 314 and automatically starts the system. The recording unit records the voice of the disaster victim using the microphone 238 of the headset terminal 314 and stores it as digital data. The dialogue unit conducts voice dialogue using the control unit 46A of the headset terminal 314 to alleviate the anxiety of the disaster victim. The transmission unit sends the stored message to family and friends using the specific processing unit 290 of the data processing unit 12. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0144] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0145] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0146] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0147] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0148] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0149] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0150] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0151] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0152] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0153] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0154] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0155] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0156] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0157] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0158] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0159] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0160] Each of the multiple elements described above, including the activation unit, recording unit, dialogue unit, and transmission unit, is implemented, for example, in at least one of the robot 414 and the data processing unit 12. For example, the activation unit uses the robot 414's sensors to detect the occurrence of a disaster and automatically activates the system. The recording unit uses the robot 414's microphone 238 to record the voices of disaster victims and stores them as digital data. The dialogue unit uses the robot 414's control unit 46A to conduct voice dialogue and alleviate the disaster victims' anxiety. The transmission unit uses the data processing unit 12's specific processing unit 290 to send messages stored by the data processing unit 12 to family and friends. The correspondence between each unit and the device or control unit is not limited to the examples described above and can be modified in various ways.

[0161] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0162] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0163] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0164] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0165] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0166] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0167] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0168] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0169] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0170] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0171] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0172] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0173] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0174] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0175] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0176] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0177] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0178] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0179] (Note 1) A startup unit that automatically activates in the event of a disaster, The Records Department records and preserves the voices and thoughts of the disaster victims, A dialogue unit that alleviates anxiety through dialogue based on the information recorded by the aforementioned recording unit, A transmitting unit that sends saved messages is included. A system characterized by the following features. (Note 2) The aforementioned startup unit is It automatically activates in the event of a disaster such as a tsunami or landslide. The system described in Appendix 1, characterized by the features described herein. (Note 3) The recording unit is, Record and preserve the voices and thoughts of disaster victims. The system described in Appendix 1, characterized by the features described herein. (Note 4) The aforementioned dialogue unit, We engage in dialogue with disaster victims and provide support to alleviate their anxieties. The system described in Appendix 1, characterized by the features described herein. (Note 5) The aforementioned transmitting unit After confirming safety, send the saved message to family and friends. The system described in Appendix 1, characterized by the features described herein. (Note 6) The aforementioned startup unit is It estimates the user's emotions and adjusts the launch timing based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 7) The aforementioned startup unit is The method of activation will be changed depending on the type and scale of the disaster. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned startup unit is Using smartphone sensor information to more accurately detect the occurrence of disasters. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned startup unit is It estimates the user's emotions and determines the launch priority based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned startup unit is It starts up in conjunction with other devices. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned startup unit is It uses geographical location information to activate only when a disaster occurs in a specific region. The system described in Appendix 1, characterized by the features described herein. (Note 12) The recording unit is, The system estimates the user's emotions and adjusts the recording method based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 13) The recording unit is, It will be possible to record not only audio, but also text and images. The system described in Appendix 1, characterized by the features described herein. (Note 14) The recording unit is, Add a feature to automatically categorize and organize the content of recorded messages. The system described in Appendix 1, characterized by the features described herein. (Note 15) The recording unit is, The system estimates the user's emotions and prioritizes recordings based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 16) The recording unit is, Recorded messages are backed up to the cloud in real time. The system described in Appendix 1, characterized by the features described herein. (Note 17) The recording unit is, Encrypt and store the recorded messages. The system described in Appendix 1, characterized by the features described herein. (Note 18) The aforementioned dialogue unit, It estimates the user's emotions and adjusts the content of the conversation based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 19) The aforementioned dialogue unit, Personalize the content of the conversation based on the user's past conversation history. The system described in Appendix 1, characterized by the features described herein. (Note 20) The aforementioned dialogue unit, Provide users with specific instructions and advice during the conversation. The system described in Appendix 1, characterized by the features described herein. (Note 21) The aforementioned dialogue unit, It estimates the user's emotions and determines the priority of the conversation based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 22) The aforementioned dialogue unit, Make the dialogue content multilingual. The system described in Appendix 1, characterized by the features described herein. (Note 23) The aforementioned dialogue unit, The content of the conversation will be provided not only in audio but also in text. The system described in Appendix 1, characterized by the features described herein. (Note 24) The aforementioned transmitting unit It estimates the user's emotions and adjusts the timing of sending messages based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 25) The aforementioned transmitting unit The system automatically selects message recipients based on the user's past contact history. The system described in Appendix 1, characterized by the features described herein. (Note 26) The aforementioned transmitting unit The system automatically checks the content and corrects inappropriate expressions before sending the message. The system described in Appendix 1, characterized by the features described herein. (Note 27) The aforementioned transmitting unit It estimates the user's emotions and determines the priority of messages based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 28) The aforementioned transmitting unit Support sending messages to multiple platforms. The system described in Appendix 1, characterized by the features described herein. (Note 29) The aforementioned transmitting unit We will add a feature that allows you to check the recipient's receipt status after sending a message. The system described in Appendix 1, characterized by the features described herein. [Explanation of symbols]

[0180] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. A startup unit that automatically activates in the event of a disaster, The Records Department records and preserves the voices and thoughts of the disaster victims, A dialogue unit that alleviates anxiety through dialogue based on the information recorded by the aforementioned recording unit, A transmitting unit that sends saved messages is included. A system characterized by the following features.

2. The aforementioned startup unit is It automatically activates in the event of a disaster such as a tsunami or landslide. The system according to feature 1.

3. The aforementioned recording unit is Record and preserve the voices and thoughts of disaster victims. The system according to feature 1.

4. The aforementioned dialogue unit, We engage in dialogue with disaster victims and provide support to alleviate their anxieties. The system according to feature 1.

5. The aforementioned transmitting unit After confirming safety, send the saved message to family and friends. The system according to feature 1.

6. The aforementioned startup unit is It estimates the user's emotions and adjusts the launch timing based on those emotions. The system according to feature 1.

7. The aforementioned startup unit is The method of activation will be changed depending on the type and scale of the disaster. The system according to feature 1.

8. The aforementioned startup unit is Using smartphone sensor information to more accurately detect the occurrence of disasters. The system according to feature 1.

9. The aforementioned startup unit is It estimates the user's emotions and determines the launch priority based on the estimated user emotions. The system according to feature 1.

10. The aforementioned startup unit is It starts up in conjunction with other devices. The system according to feature 1.

Images