System

Abstract

To provide a system capable of automatically generating a diary by using data in a cellular phone and utilizing the diary data in various forms.SOLUTION: A system includes a data collection part, a diary generation part, a storage management part, and a utilization part. The data collection unit collects data in the mobile phone. The diary generation unit generates a diary based on the data collected by the data collection unit. The storage management unit stores the diary data generated by the diary generation unit. The utilization part utilizes the diary data stored by the storage management part in various forms.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology has had the problem that it has not been possible to efficiently utilize data stored on mobile phones to automatically generate diaries, and that this diary data has not been fully utilized in a variety of ways.

[0005] The system according to the embodiment aims to automatically generate a diary using data stored in a mobile phone and to utilize the diary data in a variety of ways. [Means for solving the problem]

[0006] The system according to the embodiment includes a data collection unit, a diary generation unit, a storage management unit, and a utilization unit. The data collection unit collects data from a mobile phone. The diary generation unit generates a diary based on the data collected by the data collection unit. The storage management unit stores the diary data generated by the diary generation unit. The utilization unit utilizes the diary data stored by the storage management unit in a variety of ways. [Effects of the Invention]

[0007] The system according to the embodiment automatically generates a diary using data stored in a mobile phone, and the diary data can be utilized in a variety of ways. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 may have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The diary generation system according to the embodiment of the present invention is a system in which a generation AI automatically creates a diary using data stored in a mobile phone, and the diary data is utilized in a variety of ways. As a result, the diary generation system automatically creates a diary using data stored in a mobile phone, and the diary data can be utilized in a variety of ways.

[0029] A diary generation system according to an embodiment includes a data collection unit, a diary generation unit, a storage management unit, and a utilization unit. The data collection unit collects data from a mobile phone, such as photos, messages, location information, and calendar appointments. The data collection unit can also learn a user's behavioral patterns and automatically start collecting data based on specific events or actions. For example, a generation AI can learn a user's behavioral patterns and automatically start collecting data based on specific events (e.g., daily commute time or weekend outings). The diary generation unit generates a diary based on the collected data. For example, the diary describes the events of the day in detail by combining the date and time and location of photo captures, message content, location history, calendar appointments, and the like. The generation AI generates natural-sounding sentences based on prompts containing user instructions. The storage management unit stores the generated diary data. For example, the diary data is stored in the mobile phone or in cloud storage. The storage management unit can also automatically categorize the diary data and organize it by specific themes or events. For example, the generation AI can automatically categorize the diary data and organize it by specific themes (e.g., travel, work, family). The utilization unit utilizes the stored diary data in a variety of ways. For example, it automatically creates a memory album based on the diary data, analyzes a life log based on the diary data, and supports health management based on the diary data. The generation AI analyzes the diary data based on prompts including user instructions and generates appropriate suggestions and reports. As a result, the diary generation system according to the embodiment can automatically create a diary using data stored in a mobile phone and utilize the diary data in a variety of ways.

[0030] The data collection unit can learn the user's behavioral patterns and automatically start data collection based on events and actions. For example, the data collection unit uses a generation AI to learn the user's behavioral patterns and automatically start data collection based on specific events (e.g., daily commute time or weekend outings). For example, it automatically collects photos and messages taken during commute time. The data collection unit can also learn the user's behavioral patterns and automatically start data collection based on specific actions (e.g., exercise or meals). For example, it automatically collects heart rate data obtained during exercise and photos of meals. This makes it possible to automatically start data collection based on the user's behavioral patterns.

[0031] The data collection unit can collect the user's voice memos or video messages and reflect them in the diary content. In the data collection unit, for example, the generation AI collects the user's voice memos and reflects them in the diary content. For example, the data collection unit converts voice memos recorded by the user into text and incorporates the content into the diary. The data collection unit can also collect the user's video messages and reflect them in the diary content. For example, it can analyze video messages filmed by the user and incorporate the content into the diary. The data collection unit can also collect a combination of voice memos and video messages and reflect them in the diary content. For example, it can convert the content of the voice memos into text and write it in the diary in combination with the content of the video messages. In this way, the user's voice memos and video messages can be reflected in the diary content.

[0032] The data collection unit can also collect data from other smart devices and reflect it in the diary content. For example, the data collection unit uses a generation AI to collect data (e.g., heart rate and step count) from a smartwatch and reflect it in the diary content. For example, the data collection unit incorporates the user's exercise records and health data into the diary. The data collection unit can also collect data (e.g., room temperature and lighting conditions) from smart home devices and reflect it in the diary content. For example, environmental data about the time the user spent at home is recorded in the diary. The data collection unit can also collect and combine data from other smart devices and reflect it in the diary content. For example, exercise data from a smartwatch and environmental data from a smart home device are combined and recorded in the diary. This allows data from other smart devices to be reflected in the diary content.

[0033] The data collection unit can also collect data from the user's social media accounts and reflect online activities in the diary. For example, the data collection unit uses a generation AI to collect posting data from the user's social media accounts and reflect it in the diary content. For example, it incorporates photos and comments posted by the user into the diary. The data collection unit can also collect message data from the user's social media accounts and reflect it in the diary content. For example, it can record the content of messages received by the user in the diary. The data collection unit can also collect data from social media accounts from multiple platforms and reflect it in the diary content. For example, it can combine Facebook posting data and Twitter tweet data and record it in the diary. This allows data from the user's social media accounts to be reflected in the diary.

[0034] The diary generation unit can learn the user's past diary data and generate a diary that matches the user's writing style and preferences. For example, the diary generation unit uses a generation AI to learn the user's past diary data and generate a diary that matches the user's writing style and preferences. For example, the diary is created using expressions and phrases that the user prefers. The diary generation unit can also learn the user's past diary data and generate a diary by selecting topics and themes that match the user's preferences. For example, a diary is created based on themes that interest the user (e.g., travel or hobbies). The diary generation unit can also learn the user's past diary data and generate a diary that reflects the sentence structure and style that matches the user's writing style and preferences. For example, a diary is created that reflects the user's preferred sentence length and paragraph arrangement. This makes it possible to generate a diary that matches the user's writing style and preferences.

[0035] The diary generation unit can analyze the user's voice input and convert it from voice to text to generate a diary. For example, the diary generation unit generates a diary by using a generation AI to analyze the user's voice input and convert it from voice to text. For example, what the user said is converted directly into text and recorded in the diary. The diary generation unit can also generate a diary using expressions that match the user's writing style and preferences when analyzing the voice input and converting it from voice to text. For example, a diary is created that reflects the user's preferred phrases and expressions. The diary generation unit can also generate a diary that reflects the tone and emotion of the voice when analyzing the voice input and converting it from voice to text. For example, a diary is created that reflects what the user said emotionally as it is in the text. In this way, a diary can be generated by converting the user's voice input into text.

[0036] The diary generation unit can generate diaries in different languages ​​to accommodate international users. For example, the generation AI in the diary generation unit generates diaries in different languages ​​to accommodate international users. For example, diaries are created in multiple languages, such as English, French, and Chinese. When generating diaries in different languages, the diary generation unit can also use expressions that match the user's writing style and preferences. For example, a diary is created that reflects the user's preferred phrases and expressions. When generating diaries in different languages, the diary generation unit can also adjust the expressions taking into account the characteristics of the language and cultural background. For example, casual expressions are used in English and formal expressions are used in Chinese. This allows diaries to be generated in different languages ​​to accommodate international users.

[0037] The diary generation unit can automatically insert the user's photos and videos when generating a diary, thereby creating a visually rich diary. For example, when the generation AI generates a diary, the diary generation unit automatically inserts the user's photos to create a visually rich diary. For example, photos related to the diary content are placed in appropriate positions. The diary generation unit can also automatically insert the user's videos when generating a diary, thereby creating a visually rich diary. For example, videos related to the diary content are inserted in a playable format. The diary generation unit can also arrange and edit the photos and videos according to the user's preferences when automatically inserting them. For example, the diary generation unit inserts photos and videos by applying the user's preferred layout and filter. In this way, the user's photos and videos can be automatically inserted to create a visually rich diary.

[0038] The storage management unit can automatically classify diary data and organize it by specific theme or event. For example, the generation AI in the storage management unit automatically classifies diary data and organizes it by specific theme (e.g., travel, work, family). For example, travel-related diaries are grouped into one category. When automatically classifying diary data, the storage management unit can also classify it according to the user's preferences. For example, diaries are classified based on themes that interest the user. When automatically classifying diary data, the storage management unit can also classify it based on multiple criteria. For example, diaries are classified based on both theme and event. This allows diary data to be automatically classified and organized by specific theme or event.

[0039] The storage management unit periodically backs up diary data to prevent data loss. For example, the generation AI periodically backs up diary data to prevent data loss. For example, it automatically creates backups in cloud storage every week. Furthermore, when backing up diary data, the storage management unit can set the frequency and method to suit the user's preferences. For example, it performs backups at the frequency desired by the user. Furthermore, when backing up diary data, the storage management unit can encrypt and compress the data. For example, it encrypts the backup data and stores it in cloud storage. This allows diary data to be regularly backed up to prevent data loss.

[0040] The storage management unit can link the diary data with other cloud services to make it accessible on multiple platforms. For example, the storage management unit can link the diary data with other cloud services (e.g., Google Drive or Dropbox) to make it accessible on multiple platforms. For example, the storage management unit can allow the user to view the diary from any device. When linking the diary data with other cloud services, the storage management unit can also set a linking method that suits the user's preferences. For example, the storage management unit links the data with the cloud service of the user's choice. When linking the diary data with other cloud services, the storage management unit can also encrypt or compress the data. For example, the data to be linked can be encrypt and stored in the cloud service. This allows the diary data to be linked with other cloud services to be accessible on multiple platforms.

[0041] The storage management unit can automatically tag diary data to enable users to easily search for it. For example, the storage management unit uses a generation AI to automatically tag diary data to enable users to easily search for it. For example, it automatically generates appropriate tags based on the content of the diary. When automatically tagging diary data, the storage management unit can also tag diary data in accordance with the user's preferences. For example, it generates tags based on keywords desired by the user. When automatically tagging diary data, the storage management unit can also generate tags based on multiple criteria. For example, it generates tags by combining keyword-based tagging and a machine learning algorithm. This allows diary data to be automatically tagged to enable users to easily search for it.

[0042] The utilization unit can analyze the diary data and provide a report on the user's behavioral patterns and health condition. For example, the utilization unit uses a generation AI to analyze the diary data and provide a report on the user's behavioral patterns. For example, the utilization unit analyzes the user's daily activities and movement history to visualize the behavioral patterns. The utilization unit can also analyze the diary data and provide a report on the user's health condition. For example, the utilization unit analyzes the user's exercise records and sleep patterns to evaluate the health condition. When analyzing the diary data and providing a report on the user's behavioral patterns and health condition, the utilization unit can also create a report tailored to the user's preferences. For example, the utilization unit can highlight data that the user is interested in and include it in the report. This allows the utilization unit to analyze the diary data and provide a report on the user's behavioral patterns and health condition.

[0043] The utilization unit can suggest events and activities that match the user's hobbies and interests based on the diary data. For example, the utilization unit uses the generation AI to suggest events that match the user's hobbies and interests based on the diary data. For example, it can suggest new events based on events the user has previously participated in or themes that interest the user. The utilization unit can also suggest activities that match the user's hobbies and interests based on the diary data. For example, it can suggest new activities based on the user's favorite sports or hobbies. The utilization unit can also make suggestions that match the user's preferences when suggesting events and activities that match the user's hobbies and interests based on the diary data. For example, it can make suggestions based on themes that the user is interested in. This makes it possible to suggest events and activities that match the user's hobbies and interests based on the diary data.

[0044] The utilization unit can automatically visualize the user's life log based on the diary data and display it in graphs and charts. The utilization unit, for example, can automatically visualize the user's life log based on the diary data and display it in graphs and charts. For example, the utilization unit can display the user's daily activities and movement history in graphs. The utilization unit can also visualize the user's health data based on the diary data and display it in charts. For example, the utilization unit can display the user's exercise records and sleep patterns in charts. When visualizing the user's life log based on the diary data, the utilization unit can also set a display method that suits the user's preferences. For example, the utilization unit can highlight and display data that the user desires. This makes it possible to automatically visualize the user's life log based on the diary data and display it in graphs and charts.

[0045] The utilization unit can create a timeline for looking back on the user's past events based on the diary data. For example, the utilization unit uses a generation AI to create a timeline for looking back on the user's past events based on the diary data. For example, the utilization unit displays the user's important events and occurrences in chronological order. When creating a timeline for looking back on the user's past events based on the diary data, the utilization unit can also set a display method that suits the user's preferences. For example, events and occurrences desired by the user can be highlighted and displayed. When creating a timeline for looking back on the user's past events based on the diary data, the utilization unit can also display based on multiple criteria. For example, a timeline can be created based on both themes and events. This makes it possible to create a timeline for looking back on the user's past events based on the diary data.

[0046] The storage management unit encrypts data in real time, ensuring data security. For example, the generation AI encrypts data in real time in the storage management unit, ensuring data security. For example, diary data is encrypted and stored in cloud storage. The storage management unit can also set an encryption method that suits the user's preferences when encrypting data. For example, the encryption algorithm desired by the user is used. The storage management unit can also compress data when encrypting data. For example, the data is compressed before encryption and then stored. This allows data encryption to be performed in real time, ensuring data security.

[0047] The storage management unit can learn the user's privacy settings and automatically apply the optimal security measures. For example, the storage management unit uses a generative AI to learn the user's privacy settings and automatically apply the optimal security measures. For example, it restricts data access based on the user's settings. When learning the user's privacy settings, the storage management unit can also set a learning method that suits the user's preferences. For example, it can learn based on the user's desired privacy settings. When learning the user's privacy settings, the storage management unit can also learn based on multiple criteria. For example, it can learn based on both the scope of data sharing and access permissions. This makes it possible to learn the user's privacy settings and automatically apply the optimal security measures.

[0048] The storage management unit can automatically monitor data access logs and issue an alert if unauthorized access is detected. For example, the generation AI can automatically monitor data access logs and issue an alert if unauthorized access is detected. For example, it can notify the user when an abnormal access pattern is detected. When monitoring data access logs, the storage management unit can also set a monitoring method that suits the user's preferences. For example, it can monitor based on the user's desired monitoring criteria. When monitoring data access logs, the storage management unit can also monitor based on multiple criteria. For example, it can monitor based on both IP address records and access time records. This makes it possible to automatically monitor data access logs and issue an alert if unauthorized access is detected.

[0049] The storage management unit can provide a function that allows the scope of data use to be set in detail with the user's consent. The storage management unit, for example, provides a function that allows the generation AI to set the scope of data use in detail with the user's consent. For example, a setting can be made to share only specific data. The storage management unit can also provide a setting method that suits the user's preferences when setting the scope of data use. For example, the setting can be made based on the user's desired scope of use. The storage management unit can also set the scope of data use based on multiple criteria. For example, the setting can be made based on both the scope of data sharing and the purpose of use. This makes it possible to provide a function that allows the scope of data use to be set in detail with the user's consent.

[0050] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0051] The diary generation system can further analyze the user's health data and reflect changes in the user's health status in the diary. For example, the system can analyze the user's heart rate and sleep patterns and record changes in the user's health status in the diary. The diary generation system can also analyze the user's dietary data and reflect changes in nutritional balance in the diary. For example, the system can analyze the contents of the meals the user has eaten and record changes in nutritional balance in the diary. The diary generation system can also analyze the user's exercise data and reflect changes in exercise habits in the diary. For example, the system can analyze the type and frequency of exercise the user has performed and record changes in exercise habits in the diary. In this way, changes in the user's health status can be reflected in the diary.

[0052] The diary generation system can also automatically collect content based on the user's hobbies and interests and reflect it in the diary. For example, it can collect news articles and blog articles related to topics that interest the user and write them in the diary. The diary generation system can also collect information about events and activities that the user has participated in and reflect it in the diary. For example, it can collect information about concerts and sporting events that the user has attended and write it in the diary. The diary generation system can also collect social media posts related to topics that interest the user and reflect it in the diary. For example, it can collect posts from accounts that the user follows and write them in the diary. In this way, content based on the user's hobbies and interests can be reflected in the diary.

[0053] The diary generation system can further learn from the user's past diary data and generate a diary that matches the user's writing style and preferences. For example, the diary can be created using expressions and phrases that the user prefers. The diary generation system can also learn from the user's past diary data and generate a diary by selecting topics and themes that match the user's preferences. For example, the diary can be created based on themes that interest the user (e.g., travel or hobbies). The diary generation system can also learn from the user's past diary data and generate a diary that reflects the sentence structure and style that matches the user's writing style and preferences. For example, the diary can be created by reflecting the user's preferred sentence length and paragraph arrangement. This makes it possible to generate a diary that matches the user's writing style and preferences.

[0054] The diary generation system can also collect data from the user's social media accounts and reflect online activities in the diary. For example, it can incorporate photos and comments posted by the user into the diary. The diary generation system can also collect message data from the user's social media accounts and reflect this in the diary content. For example, it can record the contents of messages received by the user in the diary. The diary generation system can also collect data from social media accounts from multiple platforms and reflect this in the diary content. For example, it can combine Facebook post data and Twitter tweet data and record them in the diary. This allows data from the user's social media accounts to be reflected in the diary.

[0055] The diary generation system can also collect voice memos or video messages from users and reflect them in the contents of the diary. For example, the system can convert voice memos recorded by users into text and incorporate the contents into the diary. The diary generation system can also collect video messages from users and reflect them in the contents of the diary. For example, the system can analyze video messages filmed by users and incorporate the contents into the diary. The diary generation system can also collect a combination of voice memos and video messages and reflect them in the contents of the diary. For example, the contents of voice memos can be converted into text and combined with the contents of video messages and written in the diary. In this way, the user's voice memos and video messages can be reflected in the contents of the diary.

[0056] The diary generation system can also generate diaries in different languages ​​to accommodate international users. For example, diaries can be created in multiple languages, such as English, French, and Chinese. When generating diaries in different languages, the diary generation system can also use expressions that match the user's writing style and preferences. For example, a diary can be created that reflects the user's preferred phrases and expressions. When generating diaries in different languages, the diary generation system can also adjust the expressions taking into account the characteristics of the language and cultural background. For example, casual expressions can be used in English and formal expressions in Chinese. This allows diaries to be generated in different languages ​​to accommodate international users.

[0057] The processing flow of the first embodiment will be briefly explained below.

[0058] Step 1: The data collection unit collects data from the mobile phone. For example, it collects photos, messages, location information, calendar events, etc. The data collection unit can also learn the user's behavioral patterns and automatically start collecting data based on specific events or actions. For example, the generation AI can learn the user's behavioral patterns and automatically start collecting data based on specific events (e.g., daily commute time or weekend outings). Step 2: The diary generation unit generates a diary based on the collected data. For example, it combines the date and location of photos, message content, location history, calendar appointments, etc. to describe the events of the day in detail. The generation AI generates natural sentences based on prompts, including user instructions. Step 3: The storage management unit saves the generated diary data. For example, it saves the diary data on a mobile phone or in cloud storage. The storage management unit can also automatically classify the diary data and organize it by specific themes or events. For example, the generation AI automatically classifies the diary data and organizes it by specific themes (e.g., travel, work, family). Step 4: The utilization unit utilizes the saved diary data in various ways. For example, it automatically creates a memory album based on the diary data, analyzes a life log based on the diary data, and supports health management based on the diary data. The generation AI analyzes the diary data based on prompts including user instructions and generates appropriate suggestions and reports.

[0059] (Example 2) The diary generation system according to the embodiment of the present invention is a system in which a generation AI automatically creates a diary using data stored in a mobile phone, and the diary data is utilized in a variety of ways. As a result, the diary generation system automatically creates a diary using data stored in a mobile phone, and the diary data can be utilized in a variety of ways.

[0060] A diary generation system according to an embodiment includes a data collection unit, a diary generation unit, a storage management unit, and a utilization unit. The data collection unit collects data from a mobile phone, such as photos, messages, location information, and calendar appointments. The data collection unit can also learn a user's behavioral patterns and automatically start collecting data based on specific events or actions. For example, a generation AI can learn a user's behavioral patterns and automatically start collecting data based on specific events (e.g., daily commute time or weekend outings). The diary generation unit generates a diary based on the collected data. For example, the diary describes the events of the day in detail by combining the date and time and location of photo captures, message content, location history, calendar appointments, and the like. The generation AI generates natural-sounding sentences based on prompts containing user instructions. The storage management unit stores the generated diary data. For example, the diary data is stored in the mobile phone or in cloud storage. The storage management unit can also automatically categorize the diary data and organize it by specific themes or events. For example, the generation AI can automatically categorize the diary data and organize it by specific themes (e.g., travel, work, family). The utilization unit utilizes the stored diary data in a variety of ways. For example, it automatically creates a memory album based on the diary data, analyzes a life log based on the diary data, and supports health management based on the diary data. The generation AI analyzes the diary data based on prompts including user instructions and generates appropriate suggestions and reports. As a result, the diary generation system according to the embodiment can automatically create a diary using data stored in a mobile phone and utilize the diary data in a variety of ways.

[0061] The data collection unit can learn the user's behavioral patterns and automatically start data collection based on events and actions. For example, the data collection unit uses a generation AI to learn the user's behavioral patterns and automatically start data collection based on specific events (e.g., daily commute time or weekend outings). For example, it automatically collects photos and messages taken during commute time. The data collection unit can also learn the user's behavioral patterns and automatically start data collection based on specific actions (e.g., exercise or meals). For example, it automatically collects heart rate data obtained during exercise and photos of meals. This makes it possible to automatically start data collection based on the user's behavioral patterns.

[0062] The data collection unit can collect the user's voice memos or video messages and reflect them in the diary content. In the data collection unit, for example, the generation AI collects the user's voice memos and reflects them in the diary content. For example, the data collection unit converts voice memos recorded by the user into text and incorporates the content into the diary. The data collection unit can also collect the user's video messages and reflect them in the diary content. For example, it can analyze video messages filmed by the user and incorporate the content into the diary. The data collection unit can also collect a combination of voice memos and video messages and reflect them in the diary content. For example, it can convert the content of the voice memos into text and write it in the diary in combination with the content of the video messages. In this way, the user's voice memos and video messages can be reflected in the diary content.

[0063] The data collection unit can use the emotion estimation function to preferentially collect data on moments when the user shows a specific emotion. For example, the data collection unit uses the emotion estimation function to preferentially collect data on moments when the user shows a specific emotion (e.g., joy or sadness). For example, the data collection unit collects photos and messages of moments when the user shows a smile. The data collection unit can also use the emotion estimation function to preferentially collect data on moments when the user shows anger or surprise. For example, the data collection unit collects video messages and voice memos of moments when the user is surprised. The data collection unit can also use the emotion estimation function to collect data on moments when the user shows a specific emotion based on multiple emotions. For example, the data collection unit collects data on moments when the user shows joy and surprise simultaneously. This allows preferential collection of data on moments when the user shows a specific emotion.

[0064] The data collection unit can also collect data from other smart devices and reflect it in the diary content. For example, the data collection unit uses a generation AI to collect data (e.g., heart rate and step count) from a smartwatch and reflect it in the diary content. For example, the data collection unit incorporates the user's exercise records and health data into the diary. The data collection unit can also collect data (e.g., room temperature and lighting conditions) from smart home devices and reflect it in the diary content. For example, environmental data about the time the user spent at home is recorded in the diary. The data collection unit can also collect and combine data from other smart devices and reflect it in the diary content. For example, exercise data from a smartwatch and environmental data from a smart home device are combined and recorded in the diary. This allows data from other smart devices to be reflected in the diary content.

[0065] The data collection unit can also collect data from the user's social media accounts and reflect online activities in the diary. For example, the data collection unit uses a generation AI to collect posting data from the user's social media accounts and reflect it in the diary content. For example, it incorporates photos and comments posted by the user into the diary. The data collection unit can also collect message data from the user's social media accounts and reflect it in the diary content. For example, it can record the content of messages received by the user in the diary. The data collection unit can also collect data from social media accounts from multiple platforms and reflect it in the diary content. For example, it can combine Facebook posting data and Twitter tweet data and record it in the diary. This allows data from the user's social media accounts to be reflected in the diary.

[0066] The data collection unit uses the emotion estimation function to enable the generation AI to automatically start data collection when the user shows a specific emotion. For example, the data collection unit uses the emotion estimation function to enable the generation AI to automatically start data collection when the user shows a specific emotion (e.g., joy or sadness). For example, it collects photos and messages of the moment the user smiles. The data collection unit can also use the emotion estimation function to enable the generation AI to automatically start data collection when the user shows anger or surprise. For example, it collects video messages and voice notes of the moment the user was surprised. The data collection unit can also use the emotion estimation function to enable the generation AI to automatically start data collection when the user shows a specific emotion, and collect data based on multiple emotions. For example, it collects data of the moment the user shows joy and surprise simultaneously. This allows data collection to automatically start when the user shows a specific emotion.

[0067] The diary generation unit can learn the user's past diary data and generate a diary that matches the user's writing style and preferences. For example, the diary generation unit uses a generation AI to learn the user's past diary data and generate a diary that matches the user's writing style and preferences. For example, the diary is created using expressions and phrases that the user prefers. The diary generation unit can also learn the user's past diary data and generate a diary by selecting topics and themes that match the user's preferences. For example, a diary is created based on themes that interest the user (e.g., travel or hobbies). The diary generation unit can also learn the user's past diary data and generate a diary that reflects the sentence structure and style that matches the user's writing style and preferences. For example, a diary is created that reflects the user's preferred sentence length and paragraph arrangement. This makes it possible to generate a diary that matches the user's writing style and preferences.

[0068] The diary generation unit can analyze the user's voice input and convert it from voice to text to generate a diary. For example, the diary generation unit generates a diary by using a generation AI to analyze the user's voice input and convert it from voice to text. For example, what the user said is converted directly into text and recorded in the diary. The diary generation unit can also generate a diary using expressions that match the user's writing style and preferences when analyzing the voice input and converting it from voice to text. For example, a diary is created that reflects the user's preferred phrases and expressions. The diary generation unit can also generate a diary that reflects the tone and emotion of the voice when analyzing the voice input and converting it from voice to text. For example, a diary is created that reflects what the user said emotionally as it is in the text. In this way, a diary can be generated by converting the user's voice input into text.

[0069] The diary generation unit can use the emotion estimation function to generate a diary that makes extensive use of positive expressions based on the user's emotions. The diary generation unit, for example, uses the emotion estimation function to generate a diary that makes extensive use of positive expressions based on the user's emotions. For example, the diary generation unit describes moments when the user felt happy using positive expressions. Furthermore, when using the emotion estimation function to generate a diary that makes extensive use of positive expressions based on the user's emotions, the diary generation unit can also use expressions that match the user's preferences. For example, a diary is created that reflects positive phrases and expressions that the user prefers. Furthermore, when using the emotion estimation function to generate a diary that makes extensive use of positive expressions based on the user's emotions, the diary generation unit can also adjust the expressions according to the intensity and type of emotion. For example, when the user felt very happy, more emphasized positive expressions are used. This makes it possible to generate a diary that makes extensive use of positive expressions based on the user's emotions.

[0070] The diary generation unit can generate diaries in different languages ​​to accommodate international users. For example, the generation AI in the diary generation unit generates diaries in different languages ​​to accommodate international users. For example, diaries are created in multiple languages, such as English, French, and Chinese. When generating diaries in different languages, the diary generation unit can also use expressions that match the user's writing style and preferences. For example, a diary is created that reflects the user's preferred phrases and expressions. When generating diaries in different languages, the diary generation unit can also adjust the expressions taking into account the characteristics of the language and cultural background. For example, casual expressions are used in English and formal expressions are used in Chinese. This allows diaries to be generated in different languages ​​to accommodate international users.

[0071] The diary generation unit can automatically insert the user's photos and videos when generating a diary, thereby creating a visually rich diary. For example, when the generation AI generates a diary, the diary generation unit automatically inserts the user's photos to create a visually rich diary. For example, photos related to the diary content are placed in appropriate positions. The diary generation unit can also automatically insert the user's videos when generating a diary, thereby creating a visually rich diary. For example, videos related to the diary content are inserted in a playable format. The diary generation unit can also arrange and edit the photos and videos according to the user's preferences when automatically inserting them. For example, the diary generation unit inserts photos and videos by applying the user's preferred layout and filter. In this way, the user's photos and videos can be automatically inserted to create a visually rich diary.

[0072] The diary generation unit can use the emotion estimation function to automatically select a diary theme or topic based on the user's emotions. For example, the diary generation unit uses the emotion estimation function to automatically select a diary theme or topic based on the user's emotions. For example, the diary generation unit generates a diary based on the theme of an event that occurred at a moment when the user felt happy. Furthermore, the diary generation unit can use the emotion estimation function to select a theme or topic based on the user's emotions in accordance with the user's preferences. For example, a diary is created based on a theme that the user is interested in (e.g., travel or hobbies). Furthermore, the diary generation unit can use the emotion estimation function to adjust the selection based on the intensity and type of emotion when selecting a theme or topic based on the user's emotions. For example, a more positive theme is selected at a moment when the user felt very happy. In this way, the diary generation unit can automatically select a diary theme or topic based on the user's emotions.

[0073] The storage management unit can automatically classify diary data and organize it by specific theme or event. For example, the generation AI in the storage management unit automatically classifies diary data and organizes it by specific theme (e.g., travel, work, family). For example, travel-related diaries are grouped into one category. When automatically classifying diary data, the storage management unit can also classify it according to the user's preferences. For example, diaries are classified based on themes that interest the user. When automatically classifying diary data, the storage management unit can also classify it based on multiple criteria. For example, diaries are classified based on both theme and event. This allows diary data to be automatically classified and organized by specific theme or event.

[0074] The storage management unit periodically backs up diary data to prevent data loss. For example, the generation AI periodically backs up diary data to prevent data loss. For example, it automatically creates backups in cloud storage every week. Furthermore, when backing up diary data, the storage management unit can set the frequency and method to suit the user's preferences. For example, it performs backups at the frequency desired by the user. Furthermore, when backing up diary data, the storage management unit can encrypt and compress the data. For example, it encrypts the backup data and stores it in cloud storage. This allows diary data to be regularly backed up to prevent data loss.

[0075] The storage management unit uses the emotion estimation function to preferentially save diary data in which the user shows a specific emotion, and can provide a search function based on the emotion. For example, the storage management unit uses the emotion estimation function to preferentially save diary data in which the user shows a specific emotion (e.g., joy or sadness). For example, it preferentially saves diary entries in which the user felt happy. Furthermore, when using the emotion estimation function to preferentially save diary data in which the user showed a specific emotion, the storage management unit can also perform saving in accordance with the user's preferences. For example, it saves diaries based on emotions that the user considers important. Furthermore, when using the emotion estimation function to preferentially save diary data in which the user showed a specific emotion, the storage management unit can also adjust saving according to the intensity or type of the emotion. For example, it preferentially saves diary entries in which the user felt very happy. In this way, it is possible to preferentially save diary data in which the user showed a specific emotion, and to provide a search function based on the emotion.

[0076] The storage management unit can link the diary data with other cloud services to make it accessible on multiple platforms. For example, the storage management unit can link the diary data with other cloud services (e.g., Google Drive or Dropbox) to make it accessible on multiple platforms. For example, the storage management unit can allow the user to view the diary from any device. When linking the diary data with other cloud services, the storage management unit can also set a linking method that suits the user's preferences. For example, the storage management unit links the data with the cloud service of the user's choice. When linking the diary data with other cloud services, the storage management unit can also encrypt or compress the data. For example, the data to be linked can be encrypt and stored in the cloud service. This allows the diary data to be linked with other cloud services to be accessible on multiple platforms.

[0077] The storage management unit can automatically tag diary data to enable users to easily search for it. For example, the storage management unit uses a generation AI to automatically tag diary data to enable users to easily search for it. For example, it automatically generates appropriate tags based on the content of the diary. When automatically tagging diary data, the storage management unit can also tag diary data in accordance with the user's preferences. For example, it generates tags based on keywords desired by the user. When automatically tagging diary data, the storage management unit can also generate tags based on multiple criteria. For example, it generates tags by combining keyword-based tagging and a machine learning algorithm. This allows diary data to be automatically tagged to enable users to easily search for it.

[0078] The storage management unit can use the emotion estimation function to automatically highlight diary data in which the user shows a specific emotion and emphasize important events. For example, the storage management unit uses the emotion estimation function to automatically highlight diary data in which the user shows a specific emotion (e.g., joy or sadness). For example, it highlights diary entries in which the user felt happy. Furthermore, when highlighting diary data in which the user showed a specific emotion using the emotion estimation function, the storage management unit can also perform highlighting in accordance with the user's preferences. For example, it highlights diary entries based on emotions that the user finds important. Furthermore, when highlighting diary data in which the user showed a specific emotion using the emotion estimation function, the storage management unit can also adjust the highlighting according to the intensity and type of the emotion. For example, it highlights diary entries in which the user felt very happy. In this way, it is possible to automatically highlight diary data in which the user showed a specific emotion and emphasize important events.

[0079] The utilization unit can analyze the diary data and provide a report on the user's behavioral patterns and health condition. For example, the utilization unit uses a generation AI to analyze the diary data and provide a report on the user's behavioral patterns. For example, the utilization unit analyzes the user's daily activities and movement history to visualize the behavioral patterns. The utilization unit can also analyze the diary data and provide a report on the user's health condition. For example, the utilization unit analyzes the user's exercise records and sleep patterns to evaluate the health condition. When analyzing the diary data and providing a report on the user's behavioral patterns and health condition, the utilization unit can also create a report tailored to the user's preferences. For example, the utilization unit can highlight data that the user is interested in and include it in the report. This allows the utilization unit to analyze the diary data and provide a report on the user's behavioral patterns and health condition.

[0080] The utilization unit can suggest events and activities that match the user's hobbies and interests based on the diary data. For example, the utilization unit uses the generation AI to suggest events that match the user's hobbies and interests based on the diary data. For example, it can suggest new events based on events the user has previously participated in or themes that interest the user. The utilization unit can also suggest activities that match the user's hobbies and interests based on the diary data. For example, it can suggest new activities based on the user's favorite sports or hobbies. The utilization unit can also make suggestions that match the user's preferences when suggesting events and activities that match the user's hobbies and interests based on the diary data. For example, it can make suggestions based on themes that the user is interested in. This makes it possible to suggest events and activities that match the user's hobbies and interests based on the diary data.

[0081] The utilization unit can use the emotion estimation function to provide reflection and mental health support based on the user's emotions. The utilization unit, for example, uses the emotion estimation function to provide reflection based on the user's emotions. For example, it provides reflection for the user to look back on moments when they felt happy. The utilization unit can also use the emotion estimation function to provide mental health support based on the user's emotions. For example, it suggests relaxation when the user feels stressed. The utilization unit can also use the emotion estimation function to provide support tailored to the user's preferences when providing reflection and mental health support based on the user's emotions. For example, it provides the user's preferred reflection method and mental health support. In this way, the utilization unit can use the emotion estimation function to provide reflection and mental health support based on the user's emotions.

[0082] The utilization unit can automatically visualize the user's life log based on the diary data and display it in graphs and charts. The utilization unit, for example, can automatically visualize the user's life log based on the diary data and display it in graphs and charts. For example, the utilization unit can display the user's daily activities and movement history in graphs. The utilization unit can also visualize the user's health data based on the diary data and display it in charts. For example, the utilization unit can display the user's exercise records and sleep patterns in charts. When visualizing the user's life log based on the diary data, the utilization unit can also set a display method that suits the user's preferences. For example, the utilization unit can highlight and display data that the user desires. This makes it possible to automatically visualize the user's life log based on the diary data and display it in graphs and charts.

[0083] The utilization unit can create a timeline for looking back on the user's past events based on the diary data. For example, the utilization unit uses a generation AI to create a timeline for looking back on the user's past events based on the diary data. For example, the utilization unit displays the user's important events and occurrences in chronological order. When creating a timeline for looking back on the user's past events based on the diary data, the utilization unit can also set a display method that suits the user's preferences. For example, events and occurrences desired by the user can be highlighted and displayed. When creating a timeline for looking back on the user's past events based on the diary data, the utilization unit can also display based on multiple criteria. For example, a timeline can be created based on both themes and events. This makes it possible to create a timeline for looking back on the user's past events based on the diary data.

[0084] The utilization unit can use the emotion estimation function to automatically extract positive memories based on the user's emotions and provide them as reminders. For example, the utilization unit can use the emotion estimation function to automatically extract positive memories based on the user's emotions and provide them as reminders. For example, the utilization unit can provide a reminder of an event that occurred during a moment when the user felt happy. Furthermore, when using the emotion estimation function to extract positive memories based on the user's emotions, the utilization unit can also perform extraction tailored to the user's preferences. For example, the utilization unit can extract positive memories that the user finds important. Furthermore, when using the emotion estimation function to extract positive memories based on the user's emotions, the utilization unit can adjust the extraction depending on the intensity and type of the emotion. For example, the utilization unit can provide a reminder of a moment when the user felt extremely happy. In this way, the utilization unit can use the emotion estimation function to automatically extract positive memories based on the user's emotions and provide them as reminders.

[0085] The storage management unit encrypts data in real time, ensuring data security. For example, the generation AI encrypts data in real time in the storage management unit, ensuring data security. For example, diary data is encrypted and stored in cloud storage. The storage management unit can also set an encryption method that suits the user's preferences when encrypting data. For example, the encryption algorithm desired by the user is used. The storage management unit can also compress data when encrypting data. For example, the data is compressed before encryption and then stored. This allows data encryption to be performed in real time, ensuring data security.

[0086] The storage management unit can learn the user's privacy settings and automatically apply the optimal security measures. For example, the storage management unit uses a generative AI to learn the user's privacy settings and automatically apply the optimal security measures. For example, it restricts data access based on the user's settings. When learning the user's privacy settings, the storage management unit can also set a learning method that suits the user's preferences. For example, it can learn based on the user's desired privacy settings. When learning the user's privacy settings, the storage management unit can also learn based on multiple criteria. For example, it can learn based on both the scope of data sharing and access permissions. This makes it possible to learn the user's privacy settings and automatically apply the optimal security measures.

[0087] The storage management unit can use the emotion estimation function to strengthen data access restrictions when the user feels anxious. The storage management unit, for example, uses the emotion estimation function to strengthen data access restrictions when the user feels anxious. For example, the storage management unit changes data access permissions the moment the user feels anxious. Furthermore, the storage management unit can use the emotion estimation function to enforce restrictions in accordance with the user's preferences when strengthening data access restrictions when the user feels anxious. For example, the storage management unit can apply an access restriction method desired by the user. Furthermore, the storage management unit can use the emotion estimation function to adjust restrictions according to the intensity and type of emotion when strengthening data access restrictions when the user feels anxious. For example, stricter access restrictions are applied when the user feels very anxious. This allows data access restrictions to be strengthened when the user feels anxious.

[0088] The storage management unit can automatically monitor data access logs and issue an alert if unauthorized access is detected. For example, the generation AI can automatically monitor data access logs and issue an alert if unauthorized access is detected. For example, it can notify the user when an abnormal access pattern is detected. When monitoring data access logs, the storage management unit can also set a monitoring method that suits the user's preferences. For example, it can monitor based on the user's desired monitoring criteria. When monitoring data access logs, the storage management unit can also monitor based on multiple criteria. For example, it can monitor based on both IP address records and access time records. This makes it possible to automatically monitor data access logs and issue an alert if unauthorized access is detected.

[0089] The storage management unit can provide a function that allows the scope of data use to be set in detail with the user's consent. The storage management unit, for example, provides a function that allows the generation AI to set the scope of data use in detail with the user's consent. For example, a setting can be made to share only specific data. The storage management unit can also provide a setting method that suits the user's preferences when setting the scope of data use. For example, the setting can be made based on the user's desired scope of use. The storage management unit can also set the scope of data use based on multiple criteria. For example, the setting can be made based on both the scope of data sharing and the purpose of use. This makes it possible to provide a function that allows the scope of data use to be set in detail with the user's consent.

[0090] The storage management unit can use the emotion estimation function to explain the privacy policy and security measures in an easy-to-understand manner so that the user feels at ease. The storage management unit, for example, uses the emotion estimation function to explain the privacy policy and security measures in an easy-to-understand manner so that the user feels at ease. For example, when the user feels anxious, the storage management unit can provide a brief explanation of the privacy policy. The storage management unit can also use the emotion estimation function to explain the security measures in an easy-to-understand manner so that the user feels at ease. For example, when the user has a question about security, the storage management unit can explain the measures in detail. The storage management unit can also use the emotion estimation function to provide an explanation method that suits the user's preferences when explaining the privacy policy and security measures so that the user feels at ease. For example, the explanation is provided in a format desired by the user (text, video, etc.). In this way, the emotion estimation function can be used to explain the privacy policy and security measures in an easy-to-understand manner so that the user feels at ease.

[0091] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0092] The diary generation system can further analyze the user's health data and reflect changes in the user's health status in the diary. For example, the system can analyze the user's heart rate and sleep patterns and record changes in the user's health status in the diary. The diary generation system can also analyze the user's dietary data and reflect changes in nutritional balance in the diary. For example, the system can analyze the contents of the meals the user has eaten and record changes in nutritional balance in the diary. The diary generation system can also analyze the user's exercise data and reflect changes in exercise habits in the diary. For example, the system can analyze the type and frequency of exercise the user has performed and record changes in exercise habits in the diary. In this way, changes in the user's health status can be reflected in the diary.

[0093] The diary generation system can also automatically collect content based on the user's hobbies and interests and reflect it in the diary. For example, it can collect news articles and blog articles related to topics that interest the user and write them in the diary. The diary generation system can also collect information about events and activities that the user has participated in and reflect it in the diary. For example, it can collect information about concerts and sporting events that the user has attended and write it in the diary. The diary generation system can also collect social media posts related to topics that interest the user and reflect it in the diary. For example, it can collect posts from accounts that the user follows and write them in the diary. In this way, content based on the user's hobbies and interests can be reflected in the diary.

[0094] The diary generation system can further analyze the user's voice input and convert the voice to text to generate a diary. For example, what the user says is converted directly into text and written in the diary. The diary generation system can also generate a diary using expressions that match the user's writing style and preferences when analyzing the voice input and converting the voice to text. For example, a diary is created that reflects the user's preferred phrases and expressions. The diary generation system can also generate a diary that reflects the tone and emotion of the voice when analyzing the voice input and converting the voice to text. For example, a diary is created that reflects the emotional content of what the user said directly into text. In this way, a diary can be generated by converting the user's voice input into text.

[0095] The diary generation system can further generate a diary that makes extensive use of positive expressions based on the user's emotions. For example, it can describe moments when the user felt happy using positive expressions. In addition, when generating a diary that makes extensive use of positive expressions based on the user's emotions, the diary generation system can also use expressions that match the user's preferences. For example, it can create a diary that reflects the user's preferred positive phrases and expressions. In addition, when generating a diary that makes extensive use of positive expressions based on the user's emotions, the diary generation system can also adjust the expressions according to the intensity and type of emotion. For example, it can use more emphasized positive expressions for moments when the user felt very happy. In this way, it is possible to generate a diary that makes extensive use of positive expressions based on the user's emotions.

[0096] The diary generation system can also automatically select diary themes and topics based on the user's emotions. For example, a diary can be generated based on the events of moments when the user felt happy. When selecting themes and topics based on the user's emotions, the diary generation system can also select topics that match the user's preferences. For example, a diary can be created based on themes that interest the user (e.g., travel or hobbies). When selecting themes and topics based on the user's emotions, the diary generation system can also adjust the selection depending on the intensity and type of the emotion. For example, a more positive theme can be selected when the user felt very happy. This makes it possible to automatically select diary themes and topics based on the user's emotions.

[0097] The diary generation system can further learn from the user's past diary data and generate a diary that matches the user's writing style and preferences. For example, the diary can be created using expressions and phrases that the user prefers. The diary generation system can also learn from the user's past diary data and generate a diary by selecting topics and themes that match the user's preferences. For example, the diary can be created based on themes that interest the user (e.g., travel or hobbies). The diary generation system can also learn from the user's past diary data and generate a diary that reflects the sentence structure and style that matches the user's writing style and preferences. For example, the diary can be created by reflecting the user's preferred sentence length and paragraph arrangement. This makes it possible to generate a diary that matches the user's writing style and preferences.

[0098] The diary generation system can also collect data from the user's social media accounts and reflect online activities in the diary. For example, it can incorporate photos and comments posted by the user into the diary. The diary generation system can also collect message data from the user's social media accounts and reflect this in the diary content. For example, it can record the contents of messages received by the user in the diary. The diary generation system can also collect data from social media accounts from multiple platforms and reflect this in the diary content. For example, it can combine Facebook post data and Twitter tweet data and record them in the diary. This allows data from the user's social media accounts to be reflected in the diary.

[0099] The diary generation system can also collect voice memos or video messages from users and reflect them in the contents of the diary. For example, the system can convert voice memos recorded by users into text and incorporate the contents into the diary. The diary generation system can also collect video messages from users and reflect them in the contents of the diary. For example, the system can analyze video messages filmed by users and incorporate the contents into the diary. The diary generation system can also collect a combination of voice memos and video messages and reflect them in the contents of the diary. For example, the contents of voice memos can be converted into text and combined with the contents of video messages and written in the diary. In this way, the user's voice memos and video messages can be reflected in the contents of the diary.

[0100] The diary generation system can further provide reflection and mental health support based on the user's emotions. For example, it can provide reflection for the user to look back on moments when they felt happy. The diary generation system can also provide mental health support based on the user's emotions. For example, it can suggest relaxation methods when the user feels stressed. Furthermore, when providing reflection and mental health support based on the user's emotions, the diary generation system can also provide support tailored to the user's preferences. For example, it can provide reflection methods and mental health support that the user prefers. In this way, it is possible to use the emotion estimation function to provide reflection and mental health support based on the user's emotions.

[0101] The diary generation system can also generate diaries in different languages ​​to accommodate international users. For example, diaries can be created in multiple languages, such as English, French, and Chinese. When generating diaries in different languages, the diary generation system can also use expressions that match the user's writing style and preferences. For example, a diary can be created that reflects the user's preferred phrases and expressions. When generating diaries in different languages, the diary generation system can also adjust the expressions taking into account the characteristics of the language and cultural background. For example, casual expressions can be used in English and formal expressions in Chinese. This allows diaries to be generated in different languages ​​to accommodate international users.

[0102] The processing flow of the second embodiment will be briefly explained below.

[0103] Step 1: The data collection unit collects data from the mobile phone. For example, it collects photos, messages, location information, calendar events, etc. The data collection unit can also learn the user's behavioral patterns and automatically start collecting data based on specific events or actions. For example, the generation AI can learn the user's behavioral patterns and automatically start collecting data based on specific events (e.g., daily commute time or weekend outings). Step 2: The diary generation unit generates a diary based on the collected data. For example, it combines the date and location of photos, message content, location history, calendar appointments, etc. to describe the events of the day in detail. The generation AI generates natural sentences based on prompts, including user instructions. Step 3: The storage management unit saves the generated diary data. For example, it saves the diary data on a mobile phone or in cloud storage. The storage management unit can also automatically classify the diary data and organize it by specific themes or events. For example, the generation AI automatically classifies the diary data and organizes it by specific themes (e.g., travel, work, family). Step 4: The utilization unit utilizes the saved diary data in various ways. For example, it automatically creates a memory album based on the diary data, analyzes a life log based on the diary data, and supports health management based on the diary data. The generation AI analyzes the diary data based on prompts including user instructions and generates appropriate suggestions and reports.

[0104] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0105] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> Examples of generative AIs include the data generation model 58, such as a neural network model (e.g., a neural network model), and a neural network model (e.g., a neural network model). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input to the data generation model 58. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0106] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0107] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0108] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0109] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0110] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0111] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0112] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0113] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0114] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0115] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0116] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0117] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 may have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0118] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0119] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0120] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0121] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0122] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0123] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0124] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0125] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0126] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0127] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0128] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0129] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0130] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0131] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0132] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 may also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0133] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0134] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0135] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0136] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0137] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0138] 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0139] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0140] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0141] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0142] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0143] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0144] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0145] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0146] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0147] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0148] In the robot 414, the processor 46 performs the identification process. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 may have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0149] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0150] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0151] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0152] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0153] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0154] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0155] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0156] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0157] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0158] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0159] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0160] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0161] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0162] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0163] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0164] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0165] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0166] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0167] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0168] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0169] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0170] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0171] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. a data collection unit that collects data in the mobile phone; a diary creation unit that creates a diary based on the data collected by the data collection unit; a storage management unit for storing the diary data generated by the diary generation unit; a utilization unit that utilizes the diary data stored by the storage management unit in various ways. A system characterized by:

2. The data collection unit The data is collected preferentially at the moment when the user shows a specific emotion.

2. The system of claim 1.

3. The diary creation unit The system learns the user's past diary data and generates the diary according to the user's writing style and preferences.

2. The system of claim 1.

4. The storage management unit Automatically classify the diary data and organize it by specific themes or events.

2. The system of claim 1.

5. The utilization unit: The diary data is analyzed and a report is provided on the user's behavioral patterns and health status.

2. The system of claim 1.

6. The utilization unit: Providing users with emotional reflection and mental health support 2. The system of claim 1.

Images