system

Abstract

The system according to the embodiment aims to memorize the contents of past conversations and easily search and summarize necessary information. [Solution] A system according to an embodiment includes a storage unit, a search unit, a summarization unit, and a protection unit. The storage unit converts speech data into text data and stores it. The search unit searches the data stored by the storage unit. The summarization unit provides important points and summaries from the data searched by the search unit. The protection unit encrypts and deletes the data stored by the storage unit.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional techniques, it is difficult to recall details of past conversations, and there is a risk of missing important information.

[0005] The system according to the embodiment aims to memorize the contents of past conversations and easily search and summarize necessary information. [Means for solving the problem]

[0006] The system according to the embodiment includes a storage unit, a search unit, a summarization unit, and a protection unit. The storage unit converts speech data into text data and stores it. The search unit searches the data stored by the storage unit. The summarization unit provides important points and summaries from the data searched by the search unit. The protection unit encrypts and deletes the data stored by the storage unit. [Effects of the Invention]

[0007] The system according to the embodiment stores the contents of past conversations and can easily search and summarize necessary information. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) A conversation memory system according to an embodiment of the present invention uses an AI to memorize conversation content and enable users to recall past conversations in detail. In the conversation memory system, when a user converses, the AI ​​memorizes the conversation content in real time. Then, when the user later wants to recall the conversation content, the AI ​​provides the memorized content. For example, the conversation memory system memorizes the content of a user's conversation with a friend and saves the content as text data. The user can then search for past conversation content by specifying specific keywords or dates and times. For example, if the user makes a request to the AI, such as "Tell me what we talked about last Friday," the AI ​​provides the content. This allows users to recall past conversations in detail and avoid forgetting even small details. The conversation memory system can also analyze conversation content and provide key points and summaries. For example, by extracting only the important parts of a long conversation and providing them to the user, users can efficiently grasp information. This allows users to recall past conversations in detail and avoid forgetting even small details. For example, important details, such as business meetings or conversations with friends, can be quickly recalled when needed. The conversation memory system also provides a summary of the conversation and key points, allowing users to grasp information efficiently. Furthermore, it saves users the trouble of taking notes, allowing them to concentrate on the conversation.

[0029] A conversation storage system according to an embodiment includes a storage unit, a search unit, a summarization unit, and a protection unit. The storage unit converts speech data of a conversation into text data and stores the text data. Examples of speech data of a conversation include, but are not limited to, telephone conversations, face-to-face conversations, and online conversations. The storage unit converts the speech data into text data using, for example, speech recognition technology. The storage unit can also analyze the speech data and convert it into text data using natural language processing technology. The storage unit can also convert the speech data into text data using a generation AI. For example, speech recognition technology converts speech data into text data in real time. Natural language processing technology understands the context of the speech data and generates accurate text data. The generation AI receives speech data as input and outputs text data. The search unit searches data stored by the storage unit. Searches can be performed using, for example, keyword search or full-text search, but are not limited to, examples. The search unit searches stored data by inputting a specific keyword, for example. The search unit can also search stored data by specifying a specific date and time. The search unit can also search stored data using a filtering method. For example, the search unit searches for data containing specific words or phrases as a keyword search. A full-text search searches all stored data. A filtering method narrows down data based on specific conditions. The summarization unit provides important points or summaries from the stored data. Summarization is performed, for example, using a summarization algorithm, but is not limited to this example. The summarization unit extracts frequently occurring keywords to perform summarization. The summarization unit can also perform summarization based on importance evaluation criteria. The summarization unit can also perform summarization using a generation AI. For example, the summarization unit uses a summarization algorithm to perform summarization based on the length and importance of a sentence. Extracting frequently occurring keywords identifies words and phrases that frequently appear in a conversation. The generation AI takes stored data as input and outputs a summary. The protection unit encrypts or deletes stored data. Encryption is performed, for example, using an encryption algorithm, but is not limited to this example.The protection unit encrypts data using, for example, the Advanced Encryption Standard (AES). The protection unit can also encrypt data using Rivest-Shamir-Adleman (RSA). The protection unit can also encrypt data using Blowfish. For example, the protection unit strongly encrypts data using AES. RSA encrypts data using public key cryptography. Blowfish provides fast and secure encryption. Deletion is performed, for example, using a complete data deletion method, but is not limited to this example. For example, the protection unit deletes data by overwriting as a complete data deletion method. The protection unit can also manage the timing of data deletion. The protection unit can also set a data retention period and delete data after the retention period has expired. For example, the protection unit overwrites and completely deletes data. The deletion timing management involves deleting data based on specific conditions. The retention period setting involves specifying a data retention period and automatically deleting data after the period has expired. This allows the conversation memory system according to the embodiment to recall past conversations in detail and efficiently grasp information by providing key points and summaries. Some or all of the above-described processing in the protection unit may be performed using, for example, AI, or may be performed without using AI. For example, the protection unit may input stored data to a generation AI and have the generation AI encrypt or delete the data.

[0030] The storage unit can convert the voice data of a conversation into text data and store it. The storage unit can convert the voice data into text data using, for example, voice recognition technology. For example, the storage unit can automatically analyze the voice using voice recognition software and store it as text data. The storage unit can also analyze the voice data using natural language processing technology and convert it into text data. For example, the storage unit can understand the context of the voice data and generate accurate text data. The storage unit can also convert the voice data into text data using a generation AI. For example, the storage unit uses a generation AI that inputs voice data and outputs text data. By converting the voice data of a conversation into text data, it becomes easier to search and summarize later. Some or all of the above-mentioned processing in the storage unit can be performed using, for example, AI, or without AI.

[0031] The search unit can search for stored data by specifying a specific keyword or date and time. The search unit, for example, inputs a specific keyword to search for stored data. For example, the search unit searches for data containing a specific word or phrase as a keyword search. The search unit can also search for stored data by specifying a specific date and time. For example, the search unit can search for data by inputting a specific date or time. The search unit can also search for stored data using a filtering method. For example, the search unit uses a filtering method that narrows down data based on specific conditions. This allows required information to be quickly obtained by searching by specifying a specific keyword or date and time. Some or all of the above-mentioned processing in the search unit may be performed using, for example, AI, or may be performed without using AI.

[0032] The summarization unit can provide key points and summaries from the stored data. The summarization unit provides key points and summaries from the stored data using, for example, a summarization algorithm. For example, the summarization unit generates summaries based on the length and importance of sentences. The summarization unit can also generate summaries by extracting frequently occurring keywords. For example, the summarization unit identifies words and phrases that appear frequently in a conversation and generates summaries based on those words and phrases. The summarization unit can also generate summaries using a generation AI. For example, the summarization unit uses a generation AI that inputs stored data and outputs a summary. This allows information to be grasped efficiently by providing key points and summaries. Some or all of the above-described processing in the summarization unit may be performed using, for example, AI, or may be performed without using AI.

[0033] The protection unit can encrypt or delete stored data. The protection unit encrypts stored data using, for example, an encryption algorithm. For example, the protection unit encrypts data using the Advanced Encryption Standard (AES). The protection unit can also encrypt data using Rivest-Shamir-Adleman (RSA). The protection unit can also encrypt data using Blowfish. For example, the protection unit strongly encrypts data using AES. RSA encrypts data using a public key cryptosystem. Blowfish provides fast and secure encryption. The protection unit deletes stored data using, for example, a complete data deletion method. For example, the protection unit overwrites and completely deletes data. The protection unit can also manage the timing of data deletion. For example, the protection unit deletes data based on specific conditions. The protection unit can also set a data retention period and delete data after the retention period has expired. For example, the protection unit specifies a data retention period and automatically deletes data after the period has expired. In this way, data protection and management are possible by encrypting or deleting data. Some or all of the above-described processing in the protection unit may be performed using AI, for example, or may be performed without using AI.

[0034] The storage unit can emphasize and store specific keywords and phrases when converting the voice data of a conversation into text data. For example, when converting voice data into text data using voice recognition technology, the storage unit emphasizes and stores specific keywords and phrases. For example, when keywords such as "important" or "essential" are used during a conversation, the storage unit emphasizes and stores those parts. Furthermore, when a specific project name or person's name is used during a conversation, the storage unit can emphasize and store those parts. Furthermore, when a date, time, or location is mentioned during a conversation, the storage unit can emphasize and store those parts. By emphasizing and storing specific keywords and phrases, important information can be memorized without being overlooked. Some or all of the above-described processing in the storage unit may be performed using, for example, AI, or may be performed without using AI. For example, the storage unit can input voice data to a generation AI and cause the generation AI to emphasize specific keywords and phrases.

[0035] The storage unit can store the voice data of a conversation while taking into consideration the tone and speed of the speaker's voice when converting the voice data of a conversation into text data. For example, when converting the voice data of a conversation into text data using voice analysis technology, the storage unit stores the voice data while taking into consideration the tone and speed of the speaker's voice. For example, the storage unit identifies parts where the speaker emphasized the text and stores the parts while emphasizing them. The storage unit can also identify parts where the speaker spoke quickly and store those parts in detail. The storage unit can also identify parts where the speaker spoke slowly and store those parts in detail. In this way, by storing the voice data while taking into consideration the tone and speed of the speaker's voice, the nuances of the conversation can be accurately stored. Some or all of the above-described processing in the storage unit may be performed using, for example, AI, or may be performed without using AI. For example, the storage unit can input the voice data to a generation AI and cause the generation AI to perform storage while taking into consideration the tone and speed of the voice.

[0036] The storage unit can remove background sounds and noise when converting the voice data of a conversation into text data and store the data. The storage unit can remove background sounds and noise when converting the voice data of a conversation into text data using, for example, noise canceling technology. For example, the storage unit can remove background sounds (e.g., cafe noise) during a conversation and store clear voice data. The storage unit can also remove noise (e.g., wind noise) during a conversation and store clear voice data. The storage unit can also remove echoes and reverberations during a conversation and store clear voice data. In this way, by removing background sounds and noise before storing the data, clear voice data can be stored. Some or all of the above-described processing in the storage unit may be performed using, for example, AI, or may be performed without using AI. For example, the storage unit can input voice data to a generation AI and cause the generation AI to remove background sounds and noise.

[0037] The storage unit can store the voice data of a conversation in different languages ​​when converting the voice data of the conversation into text data. For example, when converting the voice data of a conversation into text data using multilingual voice recognition technology, the storage unit stores the voice data in different languages. For example, if the conversation is in English, the storage unit stores the voice data as English text data. Also, if the conversation is in Japanese, the storage unit can store the voice data as Japanese text data. Also, if the conversation is in multiple languages, the storage unit can store the voice data in different languages. This allows multilingual conversations to be managed by storing the voice data in different languages. Some or all of the above-described processing in the storage unit may be performed using, or without, AI. For example, the storage unit can input voice data to a generation AI and cause the generation AI to generate text data corresponding to different languages.

[0038] The storage unit can add speaker attribute information and store the text data when converting the speech data of a conversation into text data. For example, when converting the speech data of a conversation into text data using speech analysis technology, the storage unit adds speaker attribute information (age, gender, etc.) and stores the text data. For example, the storage unit acquires speaker age information and adds it to the text data and stores it. The storage unit can also acquire speaker gender information and add it to the text data and store it. The storage unit can also acquire speaker occupation information and add it to the text data and store it. In this way, by adding and storing speaker attribute information, the context of the conversation can be more accurately understood. Some or all of the above-mentioned processing in the storage unit may be performed using, for example, AI, or may be performed without using AI. For example, the storage unit can input speech data to a generation AI and cause the generation AI to add speaker attribute information.

[0039] The storage unit can add the conversation context when converting the voice data of a conversation into text data and store the text data. For example, when converting the voice data of a conversation into text data using voice analysis technology, the storage unit adds the conversation context (place, time, etc.) and stores the text data. For example, the storage unit acquires location information where the conversation took place and adds it to the text data and stores it. The storage unit can also acquire time information where the conversation took place and add it to the text data and store it. The storage unit can also acquire the situation where the conversation took place (for example, during a meeting, chatting at a cafe, etc.) and add it to the text data and store it. In this way, by adding and storing the conversation context, it is possible to accurately grasp the background information of the conversation. Some or all of the above-mentioned processing in the storage unit may be performed using, for example, AI, or may be performed without using AI. For example, the storage unit can input the voice data to a generation AI and cause the generation AI to add the conversation context.

[0040] The search unit can automatically complete and search for parts of related conversations when searching by specifying specific keywords or dates and times. The search unit can, for example, use natural language processing technology to automatically complete and search for parts of related conversations when searching by specifying specific keywords or dates and times. For example, if a user searches for "last week's meeting," the search unit can complete and search for all conversations related to last week's meeting. Furthermore, if a user searches for "important points," the search unit can complete and search for all conversations related to important points. Furthermore, if a user searches for "project name," the search unit can complete and search for all conversations related to the project. This allows for more accurate search results by automatically completing and searching for parts of related conversations. Some or all of the above-described processing in the search unit may be performed using, for example, AI, or may be performed without AI. For example, the search unit can input specific keywords or dates and times into a generation AI and have the generation AI complete related conversations.

[0041] The search unit can evaluate and display the reliability of search results when searching by specifying a specific keyword or date and time. For example, the search unit can evaluate and display the reliability of search results when searching by specifying a specific keyword or date and time using a reliability evaluation algorithm. For example, when a user searches for "important conversation," the search unit can evaluate the reliability of the conversation and prioritize displaying conversations with high reliability. Furthermore, when a user searches for "project name," the search unit can evaluate the reliability of conversations related to the project and prioritize displaying conversations with high reliability. Furthermore, when a user searches for "date and time," the search unit can evaluate the reliability of conversations related to the date and time and prioritize displaying conversations with high reliability. Thus, by evaluating and displaying the reliability of search results, highly reliable information can be provided. Some or all of the above-described processing in the search unit may be performed using, for example, AI, or may be performed without using AI. For example, the search unit can cause a generation AI to evaluate the reliability of search results.

[0042] When performing a search by specifying a specific keyword or date and time, the search unit can adjust the display order of search results based on the user's past search history. For example, when performing a search by specifying a specific keyword or date and time using an algorithm that analyzes past search history, the search unit adjusts the display order of search results based on the user's past search history. For example, the search unit can prioritize displaying keywords that the user has frequently searched for in the past. The search unit can also prioritize displaying conversations related to the date and time the user previously searched. The search unit can also analyze the user's past search history and prioritize displaying the most relevant search results. By adjusting the display order of search results based on the user's past search history, more relevant information can be provided. Some or all of the above-described processing in the search unit may be performed using, for example, AI, or may be performed without AI. For example, the search unit can input past search history into a generation AI and have the generation AI adjust the display order.

[0043] When searching by specifying a specific keyword or date and time, the search unit can play back a portion of the audio data to provide search results. For example, when searching by specifying a specific keyword or date and time using audio playback technology, the search unit can play back a portion of the audio data to provide search results. For example, if a user searches for "important conversation," the search unit can play back and provide audio data of the conversation. Furthermore, if a user searches for "project name," the search unit can play back and provide audio data of a conversation related to the project. Furthermore, if a user searches for "date and time," the search unit can play back and provide audio data of a conversation related to the date and time. By playing back a portion of the audio data to provide search results, information can be confirmed more intuitively. Some or all of the above-described processing in the search unit may be performed using, for example, AI, or may be performed without AI. For example, the search unit can input audio data to a generation AI and have the generation AI play the audio data.

[0044] The search unit can include related images and videos in the search results when searching by specifying a specific keyword or date and time. The search unit can include related images and videos in the search results when searching by specifying a specific keyword or date and time, for example, using image recognition technology or video analysis technology. For example, if a user searches for "important conversation," the search unit can provide images and videos related to the conversation. Furthermore, if a user searches for "project name," the search unit can provide images and videos related to the project. Furthermore, if a user searches for "date and time," the search unit can provide images and videos related to the date and time. In this way, by including related images and videos in the search results, information can be visually confirmed. Some or all of the above-described processing in the search unit may be performed using, for example, AI, or may be performed without using AI. For example, the search unit can input image and video data to a generation AI and cause the generation AI to search for related images and videos.

[0045] The search unit can optimize and display search results for different devices when searching by specifying a specific keyword or date and time. For example, when searching by specifying a specific keyword or date and time using responsive design technology, the search unit can optimize and display search results for different devices (smartphones, tablets, etc.). For example, if a user is using a smartphone, the search unit can display search results that fit the screen size. Also, if a user is using a tablet, the search unit can display search results optimized for a larger screen. Also, if a user is using a desktop, the search unit can display search results that include detailed information. In this way, by optimizing and displaying search results for different devices, search results that are easy to view on any device can be provided. Some or all of the above-described processing in the search unit may be performed using, for example, AI, or may be performed without using AI. For example, the search unit can input device information to a generation AI and cause the generation AI to display optimized search results.

[0046] The summarization unit can take into account the length of the conversation and the complexity of the content when extracting important points from the stored data. For example, the summarization unit can take into account the length of the conversation and the complexity of the content when extracting important points from the stored data using a summarization algorithm. For example, in the case of a long conversation, the summarization unit extracts and summarizes important points. In addition, in the case of a complex conversation, the summarization unit can provide a summary that focuses on the main points. In addition, in the case of a short conversation, the summarization unit can briefly summarize the entire conversation. In this way, an appropriate summary can be provided by summarizing while taking into account the length of the conversation and the complexity of the content. Some or all of the above-mentioned processing in the summarization unit may be performed, for example, using AI, or may be performed without using AI. For example, the summarization unit can input the stored data into a generation AI and have the generation AI generate a summary that takes into account the length of the conversation and the complexity of the content.

[0047] The summarization unit can take into account the speaker's intention and purpose when extracting important points from stored data. For example, the summarization unit can take into account the speaker's intention and purpose when extracting important points from stored data using a summarization algorithm. For example, the summarization unit can identify parts emphasized by the speaker and include those parts in the summary. The summarization unit can also analyze the speaker's intention and include important points in the summary. The summarization unit can also understand the speaker's purpose and provide a summary that is in line with that purpose. This allows for a more accurate summary to be provided by taking the speaker's intention and purpose into consideration when summarizing. Some or all of the above-described processing in the summarization unit may be performed using, for example, AI, or may be performed without AI. For example, the summarization unit can input stored data into a generation AI and cause the generation AI to generate a summary that takes the speaker's intention and purpose into consideration.

[0048] The summarization unit can understand the context of the conversation when extracting important points from the stored data and summarize. The summarization unit understands the context of the conversation and summarizes when extracting important points from the stored data using, for example, a summarization algorithm. For example, the summarization unit understands the context of the conversation and provides a summary that is in line with the context. The summarization unit can also provide an appropriate summary by taking into account background information of the conversation. The summarization unit can also analyze the flow of the conversation and provide a natural summary. In this way, a natural summary can be provided by summarizing by understanding the context of the conversation. Some or all of the above-mentioned processing in the summarization unit may be performed using, for example, AI, or may be performed without using AI. For example, the summarization unit can input the stored data to a generation AI and have the generation AI generate a summary that understands the context of the conversation.

[0049] The summarization unit may provide summaries in different formats when extracting important points from stored data. For example, the summarization unit may provide summaries in different formats (text, audio, video, etc.) when extracting important points from stored data using a summarization algorithm. For example, the summarization unit may provide the summary in text format so that the user can read it again later. The summarization unit may also provide the summary in audio format so that the user can listen to it again. The summarization unit may also provide the summary in video format so that the user can visually confirm it. Thus, by providing summaries in different formats, information can be provided according to the user's needs. Some or all of the above-described processing in the summarization unit may be performed using, for example, AI, or may be performed without using AI. For example, the summarization unit may input stored data to a generation AI and have the generation AI generate summaries in different formats.

[0050] The summarization unit may provide a summary by referring to related materials and literature when extracting important points from the stored data. For example, the summarization unit may provide a summary by referring to related materials and literature when extracting important points from the stored data using a summarization algorithm. For example, the summarization unit may refer to materials related to the content of the conversation and include them in the summary. The summarization unit may also refer to literature related to the content of the conversation and include them in the summary. The summarization unit may also refer to data related to the content of the conversation and include them in the summary. In this way, providing a summary by referring to related materials and literature can provide a deeper understanding. Some or all of the above-mentioned processing in the summarization unit may be performed, for example, using AI, or may be performed without using AI. For example, the summarization unit may input the stored data into a generation AI and cause the generation AI to generate a summary by referring to related materials and literature.

[0051] When extracting important points from stored data, the summarization unit can translate the content of the summary into different languages ​​and provide it. When extracting important points from stored data using, for example, translation technology, the summarization unit can translate the content of the summary into different languages ​​and provide it. For example, the summarization unit can translate the summary into English and provide it. The summarization unit can also translate the summary into Japanese and provide it. The summarization unit can also translate the summary into multiple languages ​​and provide it. This enables multilingual support by translating the content of the summary into different languages ​​and providing it. Some or all of the above-mentioned processing in the summarization unit can be performed using, for example, AI, or can be performed without using AI. For example, the summarization unit can input stored data into a generation AI and have the generation AI execute summaries translated into different languages.

[0052] The protection unit can take into consideration the importance and confidentiality of the data when encrypting or deleting stored data. For example, the protection unit can take into consideration the importance and confidentiality of the data when encrypting or deleting stored data using an algorithm that evaluates the importance and confidentiality of the data. For example, the protection unit strongly encrypts important data to ensure confidentiality. The protection unit can also standardly encrypt low-confidentiality data to process it efficiently. The protection unit can also quickly delete unnecessary data to improve the efficiency of data management. This enables appropriate data management by processing data while taking into consideration the importance and confidentiality of the data. Some or all of the above-described processing in the protection unit may be performed using, for example, AI, or may be performed without using AI. For example, the protection unit can input the importance and confidentiality of the data to the generation AI and have the generation AI perform the encryption or deletion processing.

[0053] The protection unit can take into account the user's access authority when encrypting or deleting stored data. For example, the protection unit can take into account the user's access authority when encrypting or deleting stored data using an access authority management algorithm. For example, the protection unit encrypts only data for which the user has access authority. The protection unit can also delete data for which the user does not have access authority. The protection unit can also encrypt or delete data based on the user's access authority. This enables appropriate data protection by processing data while taking into account the user's access authority. Some or all of the above-mentioned processing in the protection unit may be performed using, for example, AI, or may be performed without using AI. For example, the protection unit can input the user's access authority information to the generation AI and cause the generation AI to perform encryption or deletion processing based on the access authority.

[0054] The protection unit can take the data retention period into consideration when encrypting or deleting stored data. The protection unit can take the data retention period into consideration when encrypting or deleting stored data using, for example, a retention period management algorithm. For example, the protection unit strongly encrypts data with a long retention period. The protection unit can also standardly encrypt data with a short retention period. The protection unit can also quickly delete data whose retention period has expired. This enables appropriate data management by taking the data retention period into consideration when processing. Some or all of the above-mentioned processing in the protection unit may be performed using, for example, AI, or may be performed without using AI. For example, the protection unit can input data retention period information to the generation AI and have the generation AI perform encryption or deletion processing based on the retention period.

[0055] The protection unit can apply different encryption algorithms when encrypting or deleting stored data. For example, when encrypting or deleting stored data using multiple encryption algorithms, the protection unit applies different encryption algorithms. For example, the protection unit applies a strong encryption algorithm to important data. The protection unit can also apply a standard encryption algorithm to less confidential data. The protection unit can also apply a customized encryption algorithm to specific data. This enables appropriate data protection by applying different encryption algorithms for processing. Some or all of the above-mentioned processing in the protection unit may be performed using AI, for example, or may be performed without using AI. For example, the protection unit can input the encryption algorithm of the data to the generation AI and cause the generation AI to select the encryption algorithm to be applied.

[0056] The protection unit can automatically create and process a backup of data when encrypting or deleting stored data. The protection unit automatically creates and processes a backup of data when encrypting or deleting stored data using, for example, a backup management algorithm. For example, the protection unit automatically creates a backup before encrypting data. The protection unit can also automatically create a backup before deleting data. The protection unit can also encrypt and store backup data. This automatically creates and processes a backup of data, thereby improving data security. Some or all of the above-described processing in the protection unit may be performed using, for example, AI, or may be performed without using AI. For example, the protection unit can input the timing and method of backup creation to the generation AI and cause the generation AI to create the backup.

[0057] The protection unit may provide and process a data restoration method when encrypting or deleting stored data. The protection unit may provide and process a data restoration method when encrypting or deleting stored data using, for example, a restoration management algorithm. For example, the protection unit may provide a procedure for restoring encrypted data. The protection unit may also provide a procedure for restoring deleted data. The protection unit may also re-encrypt and store the restored data. By providing and processing a data restoration method, the security and convenience of data are improved. Some or all of the above-described processing in the protection unit may be performed using, for example, AI, or may be performed without using AI. For example, the protection unit may input the restoration procedure to the generation AI and cause the generation AI to restore the data.

[0058] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0059] The conversation storage system may further include a notification unit. The notification unit can notify a user when a new related conversation is recorded when the user searches for a specific keyword or date and time. For example, if a new conversation about Project X is recorded after the user searches for "conversations about Project X," the notification unit notifies the user of that information. The notification unit can also notify a user when an important conversation is recorded based on specific conditions set by the user. For example, if the user sets "conversations about important meetings," the notification unit can notify the user when a new conversation about the meeting is recorded. Furthermore, the notification unit can notify the user when a new conversation is recorded during a time period specified by the user. This allows the user to keep up to date with the latest information in real time without missing any important information.

[0060] The search unit can analyze the user's past search history and prioritize displaying relevant search results. For example, it can prioritize displaying keywords that the user has frequently searched for in the past. It can also prioritize displaying conversations related to the date and time of the user's past searches. It can also analyze the user's past search history and prioritize displaying the most relevant search results. This allows the user to provide more relevant information by adjusting the display order of search results based on the user's past search history.

[0061] The protection unit can take into consideration the importance and confidentiality of the data when encrypting or deleting stored data. For example, the protection unit can strongly encrypt important data to ensure confidentiality. The protection unit can also encrypt low-confidentiality data in a standard manner to efficiently process it. Furthermore, the protection unit can quickly delete unnecessary data to make data management more efficient. This allows for appropriate data management by taking into consideration the importance and confidentiality of the data when processing.

[0062] When converting the audio data of a conversation into text data, the memory unit can emphasize and store specific keywords and phrases. For example, if keywords such as "important" or "essential" appear during a conversation, the memory unit can emphasize and store those parts. Also, if a specific project name or person's name appears during a conversation, the memory unit can emphasize and store those parts. Furthermore, if a date, time, or location is mentioned during a conversation, the memory unit can emphasize and store those parts. In this way, by emphasizing and storing specific keywords and phrases, important information can be memorized without missing anything.

[0063] When converting the audio data of a conversation into text data, the storage unit can store the data while taking into consideration the tone and speed of the speaker's voice. For example, the storage unit can identify parts that the speaker emphasized and store those parts with emphasis. It can also identify parts that the speaker spoke quickly and store those parts in detail. It can also identify parts that the speaker spoke slowly and store those parts in detail. In this way, by taking into consideration the tone and speed of the speaker's voice when storing the data, the nuances of the conversation can be accurately stored.

[0064] The storage unit can remove background sounds and noise when converting the voice data of a conversation into text data and store the data. For example, the storage unit can remove background sounds during a conversation (e.g., cafe noise) and store clear voice data. It can also remove noise during a conversation (e.g., wind noise) and store clear voice data. It can also remove echoes and reverberations during a conversation and store clear voice data. In this way, by removing background sounds and noise before storing the data, clear voice data can be stored.

[0065] The processing flow of the first embodiment will be briefly explained below.

[0066] Step 1: The storage unit converts the voice data of the conversation into text data and stores it. Voice data of the conversation includes, for example, telephone conversations, face-to-face conversations, and online conversations. The storage unit converts the voice data into text data using voice recognition technology, natural language processing technology, or generative AI. For example, voice recognition technology converts voice data into text data in real time, and natural language processing technology understands the context of the voice data to generate accurate text data. Generative AI takes voice data as input and outputs text data. Step 2: The search unit searches the data stored by the storage unit. The search is performed using methods such as keyword search, full-text search, and filtering. For example, stored data can be searched by specifying a specific keyword or date and time. Filtering narrows down the data based on specific conditions. Step 3: The summarization unit provides key takeaways or summaries from the stored data. Summarization is done using summarization algorithms, frequent keyword extraction, importance metrics, or generative AI. For example, summarization algorithms summarize based on sentence length and importance, while frequent keyword extraction identifies words and phrases that frequently appear in a conversation. Generative AI takes the stored data as input and outputs a summary. Step 4: The protection unit encrypts or deletes the stored data. Encryption is performed using encryption algorithms such as AES, RSA, and Blowfish. For example, AES strongly encrypts data, while RSA encrypts data using public key cryptography. Blowfish provides fast and secure encryption. Data deletion is performed using a complete data deletion method, for example, by overwriting the data. The protection unit can also manage the timing of data deletion and automatically delete data whose retention period has expired.

[0067] (Example 2) A conversation memory system according to an embodiment of the present invention uses an AI to memorize conversation content and enable users to recall past conversations in detail. In the conversation memory system, when a user converses, the AI ​​memorizes the conversation content in real time. Then, when the user later wants to recall the conversation content, the AI ​​provides the memorized content. For example, the conversation memory system memorizes the content of a user's conversation with a friend and saves the content as text data. The user can then search for past conversation content by specifying specific keywords or dates and times. For example, if the user makes a request to the AI, such as "Tell me what we talked about last Friday," the AI ​​provides the content. This allows users to recall past conversations in detail and avoid forgetting even small details. The conversation memory system can also analyze conversation content and provide key points and summaries. For example, by extracting only the important parts of a long conversation and providing them to the user, users can efficiently grasp information. This allows users to recall past conversations in detail and avoid forgetting even small details. For example, important details, such as business meetings or conversations with friends, can be quickly recalled when needed. The conversation memory system also provides a summary of the conversation and key points, allowing users to grasp information efficiently. Furthermore, it saves users the trouble of taking notes, allowing them to concentrate on the conversation.

[0068] A conversation storage system according to an embodiment includes a storage unit, a search unit, a summarization unit, and a protection unit. The storage unit converts speech data of a conversation into text data and stores the text data. Examples of speech data of a conversation include, but are not limited to, telephone conversations, face-to-face conversations, and online conversations. The storage unit converts the speech data into text data using, for example, speech recognition technology. The storage unit can also analyze the speech data and convert it into text data using natural language processing technology. The storage unit can also convert the speech data into text data using a generation AI. For example, speech recognition technology converts speech data into text data in real time. Natural language processing technology understands the context of the speech data and generates accurate text data. The generation AI receives speech data as input and outputs text data. The search unit searches data stored by the storage unit. Searches can be performed using, for example, keyword search or full-text search, but are not limited to, examples. The search unit searches stored data by inputting a specific keyword, for example. The search unit can also search stored data by specifying a specific date and time. The search unit can also search stored data using a filtering method. For example, the search unit searches for data containing specific words or phrases as a keyword search. A full-text search searches all stored data. A filtering method narrows down data based on specific conditions. The summarization unit provides important points or summaries from the stored data. Summarization is performed, for example, using a summarization algorithm, but is not limited to this example. The summarization unit extracts frequently occurring keywords to perform summarization. The summarization unit can also perform summarization based on importance evaluation criteria. The summarization unit can also perform summarization using a generation AI. For example, the summarization unit uses a summarization algorithm to perform summarization based on the length and importance of a sentence. Extracting frequently occurring keywords identifies words and phrases that frequently appear in a conversation. The generation AI takes stored data as input and outputs a summary. The protection unit encrypts or deletes stored data. Encryption is performed, for example, using an encryption algorithm, but is not limited to this example.The protection unit encrypts data using, for example, the Advanced Encryption Standard (AES). The protection unit can also encrypt data using Rivest-Shamir-Adleman (RSA). The protection unit can also encrypt data using Blowfish. For example, the protection unit strongly encrypts data using AES. RSA encrypts data using public key cryptography. Blowfish provides fast and secure encryption. Deletion is performed, for example, using a complete data deletion method, but is not limited to this example. For example, the protection unit deletes data by overwriting as a complete data deletion method. The protection unit can also manage the timing of data deletion. The protection unit can also set a data retention period and delete data after the retention period has expired. For example, the protection unit overwrites and completely deletes data. The deletion timing management involves deleting data based on specific conditions. The retention period setting involves specifying a data retention period and automatically deleting data after the period has expired. This allows the conversation memory system according to the embodiment to recall past conversations in detail and efficiently grasp information by providing key points and summaries. Some or all of the above-described processing in the protection unit may be performed using, for example, AI, or may be performed without using AI. For example, the protection unit may input stored data to a generation AI and have the generation AI encrypt or delete the data.

[0069] The storage unit can convert the voice data of a conversation into text data and store it. The storage unit can convert the voice data into text data using, for example, voice recognition technology. For example, the storage unit can automatically analyze the voice using voice recognition software and store it as text data. The storage unit can also analyze the voice data using natural language processing technology and convert it into text data. For example, the storage unit can understand the context of the voice data and generate accurate text data. The storage unit can also convert the voice data into text data using a generation AI. For example, the storage unit uses a generation AI that inputs voice data and outputs text data. By converting the voice data of a conversation into text data, it becomes easier to search and summarize later. Some or all of the above-mentioned processing in the storage unit can be performed using, for example, AI, or without AI.

[0070] The search unit can search for stored data by specifying a specific keyword or date and time. The search unit, for example, inputs a specific keyword to search for stored data. For example, the search unit searches for data containing a specific word or phrase as a keyword search. The search unit can also search for stored data by specifying a specific date and time. For example, the search unit can search for data by inputting a specific date or time. The search unit can also search for stored data using a filtering method. For example, the search unit uses a filtering method that narrows down data based on specific conditions. This allows required information to be quickly obtained by searching by specifying a specific keyword or date and time. Some or all of the above-mentioned processing in the search unit may be performed using, for example, AI, or may be performed without using AI.

[0071] The summarization unit can provide key points and summaries from the stored data. The summarization unit provides key points and summaries from the stored data using, for example, a summarization algorithm. For example, the summarization unit generates summaries based on the length and importance of sentences. The summarization unit can also generate summaries by extracting frequently occurring keywords. For example, the summarization unit identifies words and phrases that appear frequently in a conversation and generates summaries based on those words and phrases. The summarization unit can also generate summaries using a generation AI. For example, the summarization unit uses a generation AI that inputs stored data and outputs a summary. This allows information to be grasped efficiently by providing key points and summaries. Some or all of the above-described processing in the summarization unit may be performed using, for example, AI, or may be performed without using AI.

[0072] The protection unit can encrypt or delete stored data. The protection unit encrypts stored data using, for example, an encryption algorithm. For example, the protection unit encrypts data using the Advanced Encryption Standard (AES). The protection unit can also encrypt data using Rivest-Shamir-Adleman (RSA). The protection unit can also encrypt data using Blowfish. For example, the protection unit strongly encrypts data using AES. RSA encrypts data using a public key cryptosystem. Blowfish provides fast and secure encryption. The protection unit deletes stored data using, for example, a complete data deletion method. For example, the protection unit overwrites and completely deletes data. The protection unit can also manage the timing of data deletion. For example, the protection unit deletes data based on specific conditions. The protection unit can also set a data retention period and delete data after the retention period has expired. For example, the protection unit specifies a data retention period and automatically deletes data after the period has expired. In this way, data protection and management are possible by encrypting or deleting data. Some or all of the above-described processing in the protection unit may be performed using AI, for example, or may be performed without using AI.

[0073] The memory unit can estimate the user's emotions and adjust the method for storing voice data based on the estimated user's emotions. The memory unit estimates the user's emotions using, for example, voice analysis technology. For example, the memory unit analyzes the tone and speed of the user's voice from the voice data to estimate the emotions. The memory unit can also estimate the user's emotions using facial expression recognition technology. For example, the memory unit analyzes the user's facial expressions captured with a camera to estimate the emotions. The memory unit can also estimate the user's emotions using text analysis technology. For example, the memory unit analyzes the content of the user's speech to estimate the emotions. The memory unit adjusts the method for storing voice data based on the estimated user's emotions. For example, if the user is feeling stressed, the memory unit can emphasize and store important parts of the conversation. Also, if the user is relaxed, the memory unit can store the entire conversation in detail. Also, if the user is in a hurry, the memory unit can store only the main points of the conversation. This allows for more appropriate storage by adjusting the method for storing voice data according to the user's emotions. Some or all of the above-described processing in the storage unit may be performed using, for example, AI, or may be performed without using AI. For example, the storage unit may input the user's emotion data into the generation AI and cause the generation AI to adjust the memory method based on the emotion.

[0074] The storage unit can emphasize and store specific keywords and phrases when converting the voice data of a conversation into text data. For example, when converting voice data into text data using voice recognition technology, the storage unit emphasizes and stores specific keywords and phrases. For example, when keywords such as "important" or "essential" are used during a conversation, the storage unit emphasizes and stores those parts. Furthermore, when a specific project name or person's name is used during a conversation, the storage unit can emphasize and store those parts. Furthermore, when a date, time, or location is mentioned during a conversation, the storage unit can emphasize and store those parts. By emphasizing and storing specific keywords and phrases, important information can be memorized without being overlooked. Some or all of the above-described processing in the storage unit may be performed using, for example, AI, or may be performed without using AI. For example, the storage unit can input voice data to a generation AI and cause the generation AI to emphasize specific keywords and phrases.

[0075] The storage unit can store the voice data of a conversation while taking into consideration the tone and speed of the speaker's voice when converting the voice data of a conversation into text data. For example, when converting the voice data of a conversation into text data using voice analysis technology, the storage unit stores the voice data while taking into consideration the tone and speed of the speaker's voice. For example, the storage unit identifies parts where the speaker emphasized the text and stores the parts while emphasizing them. The storage unit can also identify parts where the speaker spoke quickly and store those parts in detail. The storage unit can also identify parts where the speaker spoke slowly and store those parts in detail. In this way, by storing the voice data while taking into consideration the tone and speed of the speaker's voice, the nuances of the conversation can be accurately stored. Some or all of the above-described processing in the storage unit may be performed using, for example, AI, or may be performed without using AI. For example, the storage unit can input the voice data to a generation AI and cause the generation AI to perform storage while taking into consideration the tone and speed of the voice.

[0076] The storage unit can remove background sounds and noise when converting the voice data of a conversation into text data and store the data. The storage unit can remove background sounds and noise when converting the voice data of a conversation into text data using, for example, noise canceling technology. For example, the storage unit can remove background sounds (e.g., cafe noise) during a conversation and store clear voice data. The storage unit can also remove noise (e.g., wind noise) during a conversation and store clear voice data. The storage unit can also remove echoes and reverberations during a conversation and store clear voice data. In this way, by removing background sounds and noise before storing the data, clear voice data can be stored. Some or all of the above-described processing in the storage unit may be performed using, for example, AI, or may be performed without using AI. For example, the storage unit can input voice data to a generation AI and cause the generation AI to remove background sounds and noise.

[0077] The memory unit can estimate the user's emotions and determine the priority of conversations to be stored based on the estimated user's emotions. The memory unit estimates the user's emotions using, for example, voice analysis technology. For example, the memory unit analyzes the tone and speed of the user's voice from audio data to estimate the emotions. The memory unit can also estimate the user's emotions using facial expression recognition technology. For example, the memory unit analyzes the user's facial expressions captured with a camera to estimate the emotions. The memory unit can also estimate the user's emotions using text analysis technology. For example, the memory unit analyzes the content of the user's speech to estimate the emotions. The memory unit determines the priority of conversations to be stored based on the estimated user's emotions. For example, if the user is feeling stressed, the memory unit can prioritize storing important conversations. Also, if the user is relaxed, the memory unit can equally store all conversations. Also, if the user is in a hurry, the memory unit can prioritize storing the main points. In this way, by determining the priority of conversations based on the user's emotions, important conversations can be preferentially stored. Some or all of the above-described processing in the storage unit may be performed using, for example, AI, or may be performed without using AI. For example, the storage unit may input user emotion data to the generation AI and cause the generation AI to determine conversation priorities based on emotions.

[0078] The storage unit can store the voice data of a conversation in different languages ​​when converting the voice data of the conversation into text data. For example, when converting the voice data of a conversation into text data using multilingual voice recognition technology, the storage unit stores the voice data in different languages. For example, if the conversation is in English, the storage unit stores the voice data as English text data. Also, if the conversation is in Japanese, the storage unit can store the voice data as Japanese text data. Also, if the conversation is in multiple languages, the storage unit can store the voice data in different languages. This allows multilingual conversations to be managed by storing the voice data in different languages. Some or all of the above-described processing in the storage unit may be performed using, or without, AI. For example, the storage unit can input voice data to a generation AI and cause the generation AI to generate text data corresponding to different languages.

[0079] The storage unit can add speaker attribute information and store the text data when converting the speech data of a conversation into text data. For example, when converting the speech data of a conversation into text data using speech analysis technology, the storage unit adds speaker attribute information (age, gender, etc.) and stores the text data. For example, the storage unit acquires speaker age information and adds it to the text data and stores it. The storage unit can also acquire speaker gender information and add it to the text data and store it. The storage unit can also acquire speaker occupation information and add it to the text data and store it. In this way, by adding and storing speaker attribute information, the context of the conversation can be more accurately understood. Some or all of the above-mentioned processing in the storage unit may be performed using, for example, AI, or may be performed without using AI. For example, the storage unit can input speech data to a generation AI and cause the generation AI to add speaker attribute information.

[0080] The storage unit can add the conversation context when converting the voice data of a conversation into text data and store the text data. For example, when converting the voice data of a conversation into text data using voice analysis technology, the storage unit adds the conversation context (place, time, etc.) and stores the text data. For example, the storage unit acquires location information where the conversation took place and adds it to the text data and stores it. The storage unit can also acquire time information where the conversation took place and add it to the text data and store it. The storage unit can also acquire the situation where the conversation took place (for example, during a meeting, chatting at a cafe, etc.) and add it to the text data and store it. In this way, by adding and storing the conversation context, it is possible to accurately grasp the background information of the conversation. Some or all of the above-mentioned processing in the storage unit may be performed using, for example, AI, or may be performed without using AI. For example, the storage unit can input the voice data to a generation AI and cause the generation AI to add the conversation context.

[0081] The search unit can estimate a user's emotions and adjust the display method of search results based on the estimated user's emotions. The search unit can estimate a user's emotions using, for example, voice analysis technology. For example, the search unit can analyze the tone and speed of the user's voice from audio data to estimate the emotions. The search unit can also estimate a user's emotions using facial expression recognition technology. For example, the search unit can analyze the user's facial expressions captured with a camera to estimate the emotions. The search unit can also estimate a user's emotions using text analysis technology. For example, the search unit can analyze the content of the user's speech to estimate the emotions. The search unit adjusts the display method of search results based on the estimated user's emotions. For example, if the user is feeling stressed, the search unit can display search results that are simple and highly visible. If the user is relaxed, the search unit can display search results that include detailed information. If the user is in a hurry, the search unit can display search results that focus on the main points. In this way, by adjusting the display method of search results based on the user's emotions, more appropriate search results can be provided. Some or all of the above-described processing in the search unit may be performed using, for example, AI, or may be performed without using AI. For example, the search unit may input user emotion data into the generation AI and cause the generation AI to adjust the display method of search results based on the emotion.

[0082] The search unit can automatically complete and search for parts of related conversations when searching by specifying specific keywords or dates and times. The search unit can, for example, use natural language processing technology to automatically complete and search for parts of related conversations when searching by specifying specific keywords or dates and times. For example, if a user searches for "last week's meeting," the search unit can complete and search for all conversations related to last week's meeting. Furthermore, if a user searches for "important points," the search unit can complete and search for all conversations related to important points. Furthermore, if a user searches for "project name," the search unit can complete and search for all conversations related to the project. This allows for more accurate search results by automatically completing and searching for parts of related conversations. Some or all of the above-described processing in the search unit may be performed using, for example, AI, or may be performed without AI. For example, the search unit can input specific keywords or dates and times into a generation AI and have the generation AI complete related conversations.

[0083] The search unit can evaluate and display the reliability of search results when searching by specifying a specific keyword or date and time. For example, the search unit can evaluate and display the reliability of search results when searching by specifying a specific keyword or date and time using a reliability evaluation algorithm. For example, when a user searches for "important conversation," the search unit can evaluate the reliability of the conversation and prioritize displaying conversations with high reliability. Furthermore, when a user searches for "project name," the search unit can evaluate the reliability of conversations related to the project and prioritize displaying conversations with high reliability. Furthermore, when a user searches for "date and time," the search unit can evaluate the reliability of conversations related to the date and time and prioritize displaying conversations with high reliability. Thus, by evaluating and displaying the reliability of search results, highly reliable information can be provided. Some or all of the above-described processing in the search unit may be performed using, for example, AI, or may be performed without using AI. For example, the search unit can cause a generation AI to evaluate the reliability of search results.

[0084] When performing a search by specifying a specific keyword or date and time, the search unit can adjust the display order of search results based on the user's past search history. For example, when performing a search by specifying a specific keyword or date and time using an algorithm that analyzes past search history, the search unit adjusts the display order of search results based on the user's past search history. For example, the search unit can prioritize displaying keywords that the user has frequently searched for in the past. The search unit can also prioritize displaying conversations related to the date and time the user previously searched. The search unit can also analyze the user's past search history and prioritize displaying the most relevant search results. By adjusting the display order of search results based on the user's past search history, more relevant information can be provided. Some or all of the above-described processing in the search unit may be performed using, for example, AI, or may be performed without AI. For example, the search unit can input past search history into a generation AI and have the generation AI adjust the display order.

[0085] The search unit can estimate a user's emotions and prioritize search results based on the estimated user's emotions. The search unit can estimate a user's emotions using, for example, voice analysis technology. For example, the search unit can analyze the tone and speed of the user's voice from audio data to estimate emotions. The search unit can also estimate a user's emotions using facial expression recognition technology. For example, the search unit can analyze the user's facial expressions captured with a camera to estimate emotions. The search unit can also estimate a user's emotions using text analysis technology. For example, the search unit can analyze the content of the user's speech to estimate emotions. The search unit can prioritize search results based on the estimated user's emotions. For example, if the user is feeling stressed, the search unit can prioritize displaying important conversations. Also, if the user is relaxed, the search unit can display all conversations equally. Also, if the user is in a hurry, the search unit can prioritize displaying key points. In this way, by prioritizing search results based on the user's emotions, important information can be provided preferentially. Some or all of the above-described processing in the search unit may be performed using, for example, AI, or may be performed without using AI. For example, the search unit may input user emotion data into the generation AI and have the generation AI determine the priority of search results based on the emotion.

[0086] When searching by specifying a specific keyword or date and time, the search unit can play back a portion of the audio data to provide search results. For example, when searching by specifying a specific keyword or date and time using audio playback technology, the search unit can play back a portion of the audio data to provide search results. For example, if a user searches for "important conversation," the search unit can play back and provide audio data of the conversation. Furthermore, if a user searches for "project name," the search unit can play back and provide audio data of a conversation related to the project. Furthermore, if a user searches for "date and time," the search unit can play back and provide audio data of a conversation related to the date and time. By playing back a portion of the audio data to provide search results, information can be confirmed more intuitively. Some or all of the above-described processing in the search unit may be performed using, for example, AI, or may be performed without AI. For example, the search unit can input audio data to a generation AI and have the generation AI play the audio data.

[0087] The search unit can include related images and videos in the search results when searching by specifying a specific keyword or date and time. The search unit can include related images and videos in the search results when searching by specifying a specific keyword or date and time, for example, using image recognition technology or video analysis technology. For example, if a user searches for "important conversation," the search unit can provide images and videos related to the conversation. Furthermore, if a user searches for "project name," the search unit can provide images and videos related to the project. Furthermore, if a user searches for "date and time," the search unit can provide images and videos related to the date and time. In this way, by including related images and videos in the search results, information can be visually confirmed. Some or all of the above-described processing in the search unit may be performed using, for example, AI, or may be performed without using AI. For example, the search unit can input image and video data to a generation AI and cause the generation AI to search for related images and videos.

[0088] The search unit can optimize and display search results for different devices when searching by specifying a specific keyword or date and time. For example, when searching by specifying a specific keyword or date and time using responsive design technology, the search unit can optimize and display search results for different devices (smartphones, tablets, etc.). For example, if a user is using a smartphone, the search unit can display search results that fit the screen size. Also, if a user is using a tablet, the search unit can display search results optimized for a larger screen. Also, if a user is using a desktop, the search unit can display search results that include detailed information. In this way, by optimizing and displaying search results for different devices, search results that are easy to view on any device can be provided. Some or all of the above-described processing in the search unit may be performed using, for example, AI, or may be performed without using AI. For example, the search unit can input device information to a generation AI and cause the generation AI to display optimized search results.

[0089] The summarization unit can estimate the user's emotion and adjust the presentation style of the summary based on the estimated user's emotion. The summarization unit estimates the user's emotion using, for example, voice analysis technology. For example, the summarization unit analyzes the tone and speed of the user's voice from audio data to estimate the emotion. The summarization unit can also estimate the user's emotion using facial expression recognition technology. For example, the summarization unit analyzes the user's facial expression captured with a camera to estimate the emotion. The summarization unit can also estimate the user's emotion using text analysis technology. For example, the summarization unit analyzes the content of the user's speech to estimate the emotion. The summarization unit adjusts the presentation style of the summary based on the estimated user's emotion. For example, if the user is feeling stressed, the summarization unit can provide a simple, highly visible summary. If the user is relaxed, the summarization unit can provide a summary with detailed information. If the user is in a hurry, the summarization unit can provide a summary that focuses on the main points. In this way, by adjusting the presentation style of the summary based on the user's emotion, a more appropriate summary can be provided. Some or all of the above-described processing in the summarization unit may be performed using, for example, AI, or may be performed without using AI. For example, the summarization unit may input user emotion data into the generation AI and have the generation AI adjust the summary expression method based on the emotion.

[0090] The summarization unit can take into account the length of the conversation and the complexity of the content when extracting important points from the stored data. For example, the summarization unit can take into account the length of the conversation and the complexity of the content when extracting important points from the stored data using a summarization algorithm. For example, in the case of a long conversation, the summarization unit extracts and summarizes important points. In addition, in the case of a complex conversation, the summarization unit can provide a summary that focuses on the main points. In addition, in the case of a short conversation, the summarization unit can briefly summarize the entire conversation. In this way, an appropriate summary can be provided by summarizing while taking into account the length of the conversation and the complexity of the content. Some or all of the above-mentioned processing in the summarization unit may be performed, for example, using AI, or may be performed without using AI. For example, the summarization unit can input the stored data into a generation AI and have the generation AI generate a summary that takes into account the length of the conversation and the complexity of the content.

[0091] The summarization unit can take into account the speaker's intention and purpose when extracting important points from stored data. For example, the summarization unit can take into account the speaker's intention and purpose when extracting important points from stored data using a summarization algorithm. For example, the summarization unit can identify parts emphasized by the speaker and include those parts in the summary. The summarization unit can also analyze the speaker's intention and include important points in the summary. The summarization unit can also understand the speaker's purpose and provide a summary that is in line with that purpose. This allows for a more accurate summary to be provided by taking the speaker's intention and purpose into consideration when summarizing. Some or all of the above-described processing in the summarization unit may be performed using, for example, AI, or may be performed without AI. For example, the summarization unit can input stored data into a generation AI and cause the generation AI to generate a summary that takes the speaker's intention and purpose into consideration.

[0092] The summarization unit can understand the context of the conversation when extracting important points from the stored data and summarize. The summarization unit understands the context of the conversation and summarizes when extracting important points from the stored data using, for example, a summarization algorithm. For example, the summarization unit understands the context of the conversation and provides a summary that is in line with the context. The summarization unit can also provide an appropriate summary by taking into account background information of the conversation. The summarization unit can also analyze the flow of the conversation and provide a natural summary. In this way, a natural summary can be provided by summarizing by understanding the context of the conversation. Some or all of the above-mentioned processing in the summarization unit may be performed using, for example, AI, or may be performed without using AI. For example, the summarization unit can input the stored data to a generation AI and have the generation AI generate a summary that understands the context of the conversation.

[0093] The summarization unit can estimate the user's emotion and adjust the length of the summary based on the estimated user's emotion. The summarization unit can estimate the user's emotion using, for example, voice analysis technology. For example, the summarization unit can analyze the user's tone and speed of voice from audio data to estimate the emotion. The summarization unit can also estimate the user's emotion using facial expression recognition technology. For example, the summarization unit can analyze the user's facial expression captured with a camera to estimate the emotion. The summarization unit can also estimate the user's emotion using text analysis technology. For example, the summarization unit can analyze the content of the user's speech to estimate the emotion. The summarization unit adjusts the length of the summary based on the estimated user's emotion. For example, if the user is stressed, the summarization unit can provide a short, concise summary. If the user is relaxed, the summarization unit can provide a longer summary with detailed information. If the user is in a hurry, the summarization unit can provide a concise, concise summary. In this way, by adjusting the length of the summary based on the user's emotion, a more appropriate summary can be provided. Some or all of the above-described processing in the summarization unit may be performed using AI, or may be performed without AI. For example, the summarization unit may input user emotion data into the generation AI and have the generation AI adjust the length of the summary based on the emotion.

[0094] The summarization unit may provide summaries in different formats when extracting important points from stored data. For example, the summarization unit may provide summaries in different formats (text, audio, video, etc.) when extracting important points from stored data using a summarization algorithm. For example, the summarization unit may provide the summary in text format so that the user can read it again later. The summarization unit may also provide the summary in audio format so that the user can listen to it again. The summarization unit may also provide the summary in video format so that the user can visually confirm it. Thus, by providing summaries in different formats, information can be provided according to the user's needs. Some or all of the above-described processing in the summarization unit may be performed using, for example, AI, or may be performed without using AI. For example, the summarization unit may input stored data to a generation AI and have the generation AI generate summaries in different formats.

[0095] The summarization unit may provide a summary by referring to related materials and literature when extracting important points from the stored data. For example, the summarization unit may provide a summary by referring to related materials and literature when extracting important points from the stored data using a summarization algorithm. For example, the summarization unit may refer to materials related to the content of the conversation and include them in the summary. The summarization unit may also refer to literature related to the content of the conversation and include them in the summary. The summarization unit may also refer to data related to the content of the conversation and include them in the summary. In this way, providing a summary by referring to related materials and literature can provide a deeper understanding. Some or all of the above-mentioned processing in the summarization unit may be performed, for example, using AI, or may be performed without using AI. For example, the summarization unit may input the stored data into a generation AI and cause the generation AI to generate a summary by referring to related materials and literature.

[0096] When extracting important points from stored data, the summarization unit can translate the content of the summary into different languages ​​and provide it. When extracting important points from stored data using, for example, translation technology, the summarization unit can translate the content of the summary into different languages ​​and provide it. For example, the summarization unit can translate the summary into English and provide it. The summarization unit can also translate the summary into Japanese and provide it. The summarization unit can also translate the summary into multiple languages ​​and provide it. This enables multilingual support by translating the content of the summary into different languages ​​and providing it. Some or all of the above-mentioned processing in the summarization unit can be performed using, for example, AI, or can be performed without using AI. For example, the summarization unit can input stored data into a generation AI and have the generation AI execute summaries translated into different languages.

[0097] The protection unit can estimate a user's emotion and adjust the data encryption method based on the estimated user's emotion. The protection unit can estimate the user's emotion using, for example, voice analysis technology. For example, the protection unit can analyze the tone and speed of the user's voice from voice data to estimate the emotion. The protection unit can also estimate the user's emotion using facial expression recognition technology. For example, the protection unit can analyze the user's facial expression captured with a camera to estimate the emotion. The protection unit can also estimate the user's emotion using text analysis technology. For example, the protection unit can analyze the content of the user's speech to estimate the emotion. The protection unit adjusts the data encryption method based on the estimated user's emotion. For example, if the user is stressed, the protection unit can apply a strong encryption method. If the user is relaxed, the protection unit can apply a standard encryption method. If the user is in a hurry, the protection unit can apply a quick encryption method. This enables more appropriate data protection by adjusting the data encryption method based on the user's emotion. Some or all of the above-described processing in the protection unit may be performed using AI, or may be performed without AI. For example, the protection unit may input user emotion data to the generation AI and cause the generation AI to adjust the encryption method based on the emotion.

[0098] The protection unit can take into consideration the importance and confidentiality of the data when encrypting or deleting stored data. For example, the protection unit can take into consideration the importance and confidentiality of the data when encrypting or deleting stored data using an algorithm that evaluates the importance and confidentiality of the data. For example, the protection unit strongly encrypts important data to ensure confidentiality. The protection unit can also standardly encrypt low-confidentiality data to process it efficiently. The protection unit can also quickly delete unnecessary data to improve the efficiency of data management. This enables appropriate data management by processing data while taking into consideration the importance and confidentiality of the data. Some or all of the above-described processing in the protection unit may be performed using, for example, AI, or may be performed without using AI. For example, the protection unit can input the importance and confidentiality of the data to the generation AI and have the generation AI perform the encryption or deletion processing.

[0099] The protection unit can take into account the user's access authority when encrypting or deleting stored data. For example, the protection unit can take into account the user's access authority when encrypting or deleting stored data using an access authority management algorithm. For example, the protection unit encrypts only data for which the user has access authority. The protection unit can also delete data for which the user does not have access authority. The protection unit can also encrypt or delete data based on the user's access authority. This enables appropriate data protection by processing data while taking into account the user's access authority. Some or all of the above-mentioned processing in the protection unit may be performed using, for example, AI, or may be performed without using AI. For example, the protection unit can input the user's access authority information to the generation AI and cause the generation AI to perform encryption or deletion processing based on the access authority.

[0100] The protection unit can take the data retention period into consideration when encrypting or deleting stored data. The protection unit can take the data retention period into consideration when encrypting or deleting stored data using, for example, a retention period management algorithm. For example, the protection unit strongly encrypts data with a long retention period. The protection unit can also standardly encrypt data with a short retention period. The protection unit can also quickly delete data whose retention period has expired. This enables appropriate data management by taking the data retention period into consideration when processing. Some or all of the above-mentioned processing in the protection unit may be performed using, for example, AI, or may be performed without using AI. For example, the protection unit can input data retention period information to the generation AI and have the generation AI perform encryption or deletion processing based on the retention period.

[0101] The protection unit can estimate the user's emotions and adjust the timing of data deletion based on the estimated user's emotions. The protection unit can estimate the user's emotions using, for example, voice analysis technology. For example, the protection unit can analyze the tone and speed of the user's voice from audio data to estimate the emotions. The protection unit can also estimate the user's emotions using facial expression recognition technology. For example, the protection unit can analyze the user's facial expressions captured with a camera to estimate the emotions. The protection unit can also estimate the user's emotions using text analysis technology. For example, the protection unit can analyze the content of the user's speech to estimate the emotions. The protection unit adjusts the timing of data deletion based on the estimated user's emotions. For example, if the user is feeling stressed, the protection unit can delete data quickly. Alternatively, if the user is relaxed, the protection unit can delete data at a standard timing. Alternatively, if the user is in a hurry, the protection unit can delete data immediately. This enables appropriate data management by adjusting the timing of data deletion based on the user's emotions. Some or all of the above-described processing in the protection unit may be performed using, for example, AI, or may be performed without using AI. For example, the protection unit can input the user's emotional data into the generation AI and cause the generation AI to adjust the deletion timing based on the emotion.

[0102] The protection unit can apply different encryption algorithms when encrypting or deleting stored data. For example, when encrypting or deleting stored data using multiple encryption algorithms, the protection unit applies different encryption algorithms. For example, the protection unit applies a strong encryption algorithm to important data. The protection unit can also apply a standard encryption algorithm to less confidential data. The protection unit can also apply a customized encryption algorithm to specific data. This enables appropriate data protection by applying different encryption algorithms for processing. Some or all of the above-mentioned processing in the protection unit may be performed using AI, for example, or may be performed without using AI. For example, the protection unit can input the encryption algorithm of the data to the generation AI and cause the generation AI to select the encryption algorithm to be applied.

[0103] The protection unit can automatically create and process a backup of data when encrypting or deleting stored data. The protection unit automatically creates and processes a backup of data when encrypting or deleting stored data using, for example, a backup management algorithm. For example, the protection unit automatically creates a backup before encrypting data. The protection unit can also automatically create a backup before deleting data. The protection unit can also encrypt and store backup data. This automatically creates and processes a backup of data, thereby improving data security. Some or all of the above-described processing in the protection unit may be performed using, for example, AI, or may be performed without using AI. For example, the protection unit can input the timing and method of backup creation to the generation AI and cause the generation AI to create the backup.

[0104] The protection unit may provide and process a data restoration method when encrypting or deleting stored data. The protection unit may provide and process a data restoration method when encrypting or deleting stored data using, for example, a restoration management algorithm. For example, the protection unit may provide a procedure for restoring encrypted data. The protection unit may also provide a procedure for restoring deleted data. The protection unit may also re-encrypt and store the restored data. By providing and processing a data restoration method, the security and convenience of data are improved. Some or all of the above-described processing in the protection unit may be performed using, for example, AI, or may be performed without using AI. For example, the protection unit may input the restoration procedure to the generation AI and cause the generation AI to restore the data. === Hard Collateral 1-1 === Each of the multiple elements, including the storage unit, search unit, summarization unit, and protection unit, described above, is implemented, for example, by at least one of the smart device 14 and the data processing device 12. For example, the storage unit is implemented by the control unit 46A of the smart device 14 and converts voice data into text data and stores it. The search unit is implemented, for example, by the specific processing unit 290 of the data processing device 12 and searches stored data. The summarization unit is implemented, for example, by the specific processing unit 290 of the data processing device 12 and provides important points or summaries from the stored data. The protection unit is implemented, for example, by the control unit 46A of the smart device 14 and encrypts or deletes stored data. === Hard Collateral 1-2 === Each of the multiple elements, including the storage unit, search unit, summarization unit, and protection unit, described above, is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the storage unit is realized by the control unit 46A of the smart glasses 214 and converts voice data into text data and stores it. The search unit is realized, for example, by the specific processing unit 290 of the data processing device 12 and searches stored data. The summarization unit is realized, for example, by the specific processing unit 290 of the data processing device 12 and provides important points or summaries from the stored data. The protection unit is realized, for example, by the control unit 46A of the smart glasses 214 and encrypts or deletes stored data. === Hard Collateral 1-3 === Each of the multiple elements including the above-mentioned storage unit, search unit, summarization unit, and protection unit is realized, for example, by at least one of the headset type terminal 314 and the data processing device 12. For example, the storage unit is realized by the control unit 46A of the headset type terminal 314 and converts voice data into text data and stores it. The search unit is realized, for example, by the specific processing unit 290 of the data processing device 12 and searches stored data. The summarization unit is realized, for example, by the specific processing unit 290 of the data processing device 12 and provides important points or summaries from the stored data. The protection unit is realized, for example, by the control unit 46A of the headset type terminal 314 and encrypts or deletes stored data. === Hard Collateral 1-4 === Each of the multiple elements including the storage unit, search unit, summarization unit, and protection unit described above is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the storage unit is realized by the control unit 46A of the robot 414 and converts voice data into text data and stores it. The search unit is realized, for example, by the specific processing unit 290 of the data processing device 12 and searches stored data. The summarization unit is realized, for example, by the specific processing unit 290 of the data processing device 12 and provides important points or summaries from the stored data. The protection unit is realized, for example, by the control unit 46A of the robot 414 and encrypts or deletes stored data.

[0105] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0106] The conversation storage system may further include a notification unit. The notification unit can notify a user when a new related conversation is recorded when the user searches for a specific keyword or date and time. For example, if a new conversation about Project X is recorded after the user searches for "conversations about Project X," the notification unit notifies the user of that information. The notification unit can also notify a user when an important conversation is recorded based on specific conditions set by the user. For example, if the user sets "conversations about important meetings," the notification unit can notify the user when a new conversation about the meeting is recorded. Furthermore, the notification unit can notify the user when a new conversation is recorded during a time period specified by the user. This allows the user to keep up to date with the latest information in real time without missing any important information.

[0107] When converting the audio data of a conversation into text data, the storage unit can identify the emotional tone of the conversation and store the text data by adding an emotion tag. For example, if the user is angry in a conversation, the storage unit can add an "anger" tag to that part and store it. If the user is happy, the storage unit can add an "happiness" tag to that part and store it. Furthermore, if the user is sad, the storage unit can add an "sadness" tag to that part and store it. This allows the emotion tag to be used to quickly find conversations related to specific emotions during later searches.

[0108] The search unit can analyze the user's past search history and prioritize displaying relevant search results. For example, it can prioritize displaying keywords that the user has frequently searched for in the past. It can also prioritize displaying conversations related to the date and time of the user's past searches. It can also analyze the user's past search history and prioritize displaying the most relevant search results. This allows the user to provide more relevant information by adjusting the display order of search results based on the user's past search history.

[0109] The summarization unit can estimate the user's emotions and adjust the way the summary is presented based on the estimated user's emotions. For example, if the user is feeling stressed, the summarization unit can provide a simple, highly visible summary. If the user is relaxed, the summarization unit can also provide a summary that includes detailed information. Furthermore, if the user is in a hurry, the summarization unit can also provide a summary that focuses on the main points. In this way, by adjusting the way the summary is presented based on the user's emotions, a more appropriate summary can be provided.

[0110] The protection unit can take into consideration the importance and confidentiality of the data when encrypting or deleting stored data. For example, the protection unit can strongly encrypt important data to ensure confidentiality. The protection unit can also encrypt low-confidentiality data in a standard manner to efficiently process it. Furthermore, the protection unit can quickly delete unnecessary data to make data management more efficient. This allows for appropriate data management by taking into consideration the importance and confidentiality of the data when processing.

[0111] The storage unit can estimate the user's emotions and adjust the method of storing voice data based on the estimated user emotions. For example, if the user is feeling stressed, the storage unit can emphasize and store important parts of a conversation. If the user is relaxed, the storage unit can store the entire conversation in detail. Furthermore, if the user is in a hurry, the storage unit can store only the main points of the conversation. This allows for more appropriate storage by adjusting the method of storing voice data according to the user's emotions.

[0112] When converting the audio data of a conversation into text data, the memory unit can emphasize and store specific keywords and phrases. For example, if keywords such as "important" or "essential" appear during a conversation, the memory unit can emphasize and store those parts. Also, if a specific project name or person's name appears during a conversation, the memory unit can emphasize and store those parts. Furthermore, if a date, time, or location is mentioned during a conversation, the memory unit can emphasize and store those parts. In this way, by emphasizing and storing specific keywords and phrases, important information can be memorized without missing anything.

[0113] When converting the audio data of a conversation into text data, the storage unit can store the data while taking into consideration the tone and speed of the speaker's voice. For example, the storage unit can identify parts that the speaker emphasized and store those parts with emphasis. It can also identify parts that the speaker spoke quickly and store those parts in detail. It can also identify parts that the speaker spoke slowly and store those parts in detail. In this way, by taking into consideration the tone and speed of the speaker's voice when storing the data, the nuances of the conversation can be accurately stored.

[0114] The storage unit can remove background sounds and noise when converting the voice data of a conversation into text data and store the data. For example, the storage unit can remove background sounds during a conversation (e.g., cafe noise) and store clear voice data. It can also remove noise during a conversation (e.g., wind noise) and store clear voice data. It can also remove echoes and reverberations during a conversation and store clear voice data. In this way, by removing background sounds and noise before storing the data, clear voice data can be stored.

[0115] The storage unit can estimate the user's emotions and determine the priority of conversations to be stored based on the estimated user's emotions. For example, if the user is feeling stressed, the storage unit can store important conversations with priority. Also, if the user is relaxed, the storage unit can store all conversations equally. Furthermore, if the user is in a hurry, the storage unit can store the main points with priority. In this way, by determining the priority of conversations based on the user's emotions, important conversations can be stored with priority.

[0116] The processing flow of the second embodiment will be briefly explained below.

[0117] Step 1: The storage unit converts the voice data of the conversation into text data and stores it. Voice data of the conversation includes, for example, telephone conversations, face-to-face conversations, and online conversations. The storage unit converts the voice data into text data using voice recognition technology, natural language processing technology, or generative AI. For example, voice recognition technology converts voice data into text data in real time, and natural language processing technology understands the context of the voice data to generate accurate text data. Generative AI takes voice data as input and outputs text data. Step 2: The search unit searches the data stored by the storage unit. The search is performed using methods such as keyword search, full-text search, and filtering. For example, stored data can be searched by specifying a specific keyword or date and time. Filtering narrows down the data based on specific conditions. Step 3: The summarization unit provides key takeaways or summaries from the stored data. Summarization is done using summarization algorithms, frequent keyword extraction, importance metrics, or generative AI. For example, summarization algorithms summarize based on sentence length and importance, while frequent keyword extraction identifies words and phrases that frequently appear in a conversation. Generative AI takes the stored data as input and outputs a summary. Step 4: The protection unit encrypts or deletes the stored data. Encryption is performed using encryption algorithms such as AES, RSA, and Blowfish. For example, AES strongly encrypts data, while RSA encrypts data using public key cryptography. Blowfish provides fast and secure encryption. Data deletion is performed using a complete data deletion method, for example, by overwriting the data. The protection unit can also manage the timing of data deletion and automatically delete data whose retention period has expired.

[0118] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0119] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AIs include the data generation model 58, such as a neural network model (e.g., a neural network model), and a neural network model (e.g., a neural network model). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input to the data generation model 58. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0120] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0121] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0122] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0123] 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0124] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0125] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0126] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0127] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0128] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0129] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0130] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0131] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0132] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0133] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0134] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0135] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0136] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0137] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0138] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0139] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0140] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0141] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0142] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0143] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0144] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0145] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0146] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0147] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0148] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0149] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0150] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0151] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0152] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0153] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0154] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0155] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0156] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0157] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0158] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0159] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0160] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0161] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0162] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0163] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0164] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0165] In the robot 414, the processor 46 performs the identification process. The storage 50 stores the identification program 60. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as the control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform the same process as the identification processing unit 290 using these models.

[0166] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0167] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0168] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0169] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0170] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0171] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0172] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0173] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0174] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0175] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0176] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0177] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0178] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0179] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0180] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0181] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0182] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0183] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0184] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0185] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0186] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0187] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0188] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0189] [Explanation of symbols]

[0190] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. a storage unit that converts the voice data of the conversation into text data and stores the text data; a search unit that searches for data stored in the storage unit; a summarizing unit that provides key points or summaries from the data retrieved by the retrieval unit; a protection unit that encrypts or deletes the data stored by the storage unit. A system characterized by:

2. The storage unit Converts the voice data of the conversation into text data and stores it 2. The system of claim 1.

3. The search unit Search for stored data by specifying a specific keyword or date and time 2. The system of claim 1.

4. The summary section Providing key points and summaries from stored data 2. The system of claim 1.

5. The protective part is Encrypt or delete stored data 2. The system of claim 1.

6. The storage unit The method estimates the user's emotion and adjusts the method for storing voice data based on the estimated user's emotion.

2. The system of claim 1.

7. The storage unit When converting the audio data of a conversation into text data, specific keywords and phrases are emphasized and memorized.

2. The system of claim 1.

8. The storage unit When converting conversational audio data into text data, the speaker's tone and speed are taken into account when memorizing the data.

2. The system of claim 1.

9. The storage unit When converting conversational audio data into text data, background sounds and noise are removed and memorized.

2. The system of claim 1.

10. The storage unit Estimate the user's emotions and prioritize conversations to be memorized based on the estimated user emotions.

2. The system of claim 1.

Images