System

Abstract

To provide a system which grasps life situation or health status of a family member apart from the family and gives a trigger of required support and communication.SOLUTION: A system comprises: a collection part; an analysis part; and a notification part. The collection part collects voice data in a house where a family member lives. The analysis part analyzes the voice data collected by the collection part through a generative AI. The notification part performs notification based on the analysis result by the analysis part.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional technology, it was difficult to understand the living conditions and health status of family members who lived far away.

[0005] The system according to the embodiment aims to grasp the living conditions and health conditions of family members living far away, and to provide necessary support and opportunities for communication. [Means for solving the problem]

[0006] The system according to the embodiment includes a collection unit, an analysis unit, and a notification unit. The collection unit collects voice data from within a house where a family lives. The analysis unit analyzes the voice data collected by the collection unit using a generation AI. The notification unit issues a notification based on the analysis result obtained by the analysis unit. Effect of the Invention

[0007] The system according to the embodiment can grasp the living conditions and health conditions of family members living far away, and provide necessary support and opportunities for communication. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The living situation grasping system according to the embodiment of the present invention is a system for grasping the living situation of family members living apart. In this system, a voice sensor installed in a house collects voice data, and a generation AI analyzes the voice data to judge the living situation of the family members. For example, it judges whether the family members are living well, seem lonely, or are at risk of illness, depression, or mental abnormality. Based on the judgment result, a notification is sent to the relevant person (basically a relative). This notification is used as an opportunity to provide some help in advance or as a means of communication. For example, the living situation grasping system installs a voice sensor in a house. This voice sensor collects voice data in the house in real time. For example, it collects voice data such as talking, laughter, crying, and coughing. This voice data is sent to the generation AI. Next, the generation AI analyzes the voice data. The generation AI analyzes the voice data and judges the living situation of the family members. For example, it judges whether the family members are living well, seem lonely, or are at risk of illness, depression, or mental abnormality by analyzing the tone and frequency of the talking, the presence or absence of laughter, and the frequency of crying. Based on the judgment result, a notification is sent to the relevant person. For example, if the generation AI judges that the person looks lonely, a notification that "the person looks lonely" is sent to the relatives. This notification is used as an opportunity for the relatives to provide some help in advance or as a means of communication. This system makes it possible to grasp the living conditions of family members living apart in real time, and to provide necessary help and communication quickly. For example, if a parent lives alone, the children can grasp the living conditions of the parent and provide necessary help quickly. Also, if a child lives alone, the parent can grasp the living conditions of the child and provide necessary help quickly. In this way, the living conditions grasping system can grasp the living conditions of family members living apart in real time, and provide necessary help and communication quickly.

[0029] The living situation grasping system according to the embodiment includes a collection unit, an analysis unit, and a notification unit. The collection unit collects voice data in a house where a family lives. The voice data includes, for example, talking, laughing, crying, coughing, etc., but is not limited to such examples. The collection unit collects voice data such as talking and laughing in real time, for example, using a voice sensor installed in the house. The collection unit can also collect data from other sensors (such as a motion sensor and a temperature sensor). For example, the collection unit can detect the movement of family members using a motion sensor and collect the data. The collection unit can also collect temperature data in the house using a temperature sensor. The analysis unit uses a generation AI to analyze the voice data collected by the collection unit. The analysis is performed, for example, using a voice recognition technology or an emotion analysis algorithm. For example, the analysis unit analyzes the tone and frequency of the talking, the presence or absence of laughter, the frequency of crying, etc., and judges whether the person is living a healthy life, seems lonely, or is at risk of illness, depression, or mental abnormality. The analysis unit can use a generation AI to analyze the voice data and determine the living conditions of the family. For example, the analysis unit can perform analysis using a generation AI model that takes voice data as input and outputs whether the family is living well, whether they look lonely, and whether they are at risk of illness, depression, or mental abnormality. The notification unit issues a notification based on the analysis result obtained by the analysis unit. The notification is performed, for example, by email notification, app notification, voice notification, or other methods. For example, the notification unit can send a notification to the relatives saying "I look lonely" based on the analysis result. The notification unit can also specifically generate the notification content. For example, the notification unit can generate specific notification content based on the analysis result and send it to the relatives. As a result, the living situation grasping system according to the embodiment can grasp the living conditions of family members living apart in real time and can quickly provide necessary help and communication. A part or all of the above-mentioned processing in the notification unit may be performed, for example, using AI, or may be performed without using AI. For example, the notification unit can generate the notification content using an AI model that takes the analysis result as input and outputs the notification content.

[0030] The collection unit collects voice data. The voice data includes, for example, talking, laughing, crying, coughing, and the like, but is not limited to such examples. For example, the collection unit uses a voice sensor installed in the house to collect voice data such as talking and laughing in real time. The voice sensor is installed in each room of the house and can capture the living sounds of the family with high sensitivity. This makes it possible to grasp the living conditions of the family in detail. The collection unit can also collect data from other sensors (motion sensors, temperature sensors, etc.). For example, the collection unit can detect the movement of the family using a motion sensor and collect the data. The motion sensor is installed in the main passages and rooms in the house and monitors the movement and activities of the family in real time. This makes it possible to grasp which room the family is in and what activities they are doing. The collection unit can also collect temperature data in the house using a temperature sensor. The temperature sensor is installed in each room of the house and monitors the temperature change in each room in real time. This makes it possible to confirm whether the family is living in a comfortable environment. Furthermore, the collection unit can centrally manage these data and cooperate with other systems and departments as necessary. For example, the collected data is stored in a cloud server and made accessible to the analysis and notification units. In addition, by adjusting the frequency and accuracy of data collection, flexible responses to specific situations and conditions are possible. This allows the collection unit to collect data efficiently and effectively, improving the performance of the entire system.

[0031] The analysis unit uses the generation AI to analyze the voice data collected by the collection unit. The analysis is performed, for example, using voice recognition technology and emotion analysis algorithms. Specifically, the content of the speech is converted into text using voice recognition technology, and emotion analysis is performed based on the text data. The emotion analysis algorithm analyzes the tone and frequency of the speech, the presence or absence of laughter, the frequency of crying, etc., and judges whether the family is living happily, seems lonely, or is at risk of illness, depression, or mental abnormality. The analysis unit can analyze the voice data using the generation AI and judge the family's living situation. For example, the analysis unit can perform analysis using a generation AI model that takes voice data as input and outputs whether the family is living happily, seems lonely, or is at risk of illness, depression, or mental abnormality. The generation AI model has learned a large amount of voice data and its analysis results, and can perform highly accurate analysis. In addition, the analysis unit can also evaluate long-term changes in living conditions by utilizing past data and statistical information. For example, based on past voice data, it analyzes changes in family members' emotions and changes in their living patterns to predict future risks. In addition, the analysis unit can use anomaly detection algorithms to detect unusual patterns or abnormal data and issue early warnings, allowing the analysis unit to not only grasp the situation in real time, but also handle long-term risk management and anomaly detection, improving the reliability and safety of the entire system.

[0032] The notification unit issues a notification based on the analysis result obtained by the analysis unit. The notification is, for example, by email notification, app notification, voice notification, or the like. Specifically, based on the analysis result, a notification such as "I look lonely" can be sent to the relative. The notification unit can generate specific notification content based on the analysis result and send it to the relative. For example, if the analysis result is determined to be "I look lonely," the notification unit generates specific notification content such as "I look lonely recently. Please call me." and sends it to the relative. The notification unit can also customize the notification content. For example, the details and frequency of the notification content can be adjusted according to the relative's wishes. Furthermore, the notification unit can reliably transmit information using multiple communication means. For example, important information can be reliably delivered by using not only smartphone notifications but also voice calls, SMS, emails, and the like. This allows the notification unit to quickly and reliably provide action instructions to the user and minimize the risk of disaster. Furthermore, the notification unit can collect user feedback and continuously improve the accuracy and effectiveness of the notification content. For example, the notification content is reviewed and improved based on feedback from the relative who received the notification. This enables the notification unit to always provide highly accurate notifications based on the latest information, and to assist in prompt and appropriate responses.

[0033] The collection unit can collect voice data such as talking, laughter, crying, coughing, and other voice data. The collection unit collects voice data such as talking and laughter in real time, for example, using a voice sensor installed in the house. For example, the collection unit collects the tone and frequency of the talking voice to grasp the living conditions of the family. The collection unit can also collect the presence or absence of laughter to grasp the emotional state of the family. Furthermore, the collection unit can collect the frequency of crying voice to grasp the stress state of the family. For example, the collection unit can determine that the family is living a healthy life when the tone of the talking voice is high. Furthermore, the collection unit can determine that the family is relaxed when there is a lot of laughter. Furthermore, the collection unit can determine that the family is feeling stressed when there is a lot of crying. In this way, by collecting various voice data, the living conditions of the family can be grasped in detail. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using AI, or may be performed without using AI. For example, the collection unit can input the voice data acquired by the voice sensor to the generation AI and cause the generation AI to execute analysis of the voice data.

[0034] The analysis unit can analyze the voice data and determine whether the family members are living well, feeling lonely, or at risk of illness, depression, or mental abnormality. The analysis unit, for example, uses voice recognition technology to analyze the tone and frequency of the speech. For example, if the tone of the speech is high, the analysis unit can determine that the family members are living well. Also, if the tone of the speech is low, the analysis unit can determine that the family members feel lonely. Also, the analysis unit can analyze the presence or absence of laughter using an emotion analysis algorithm. For example, if there is a lot of laughter, the analysis unit can determine that the family members are relaxed. Also, the analysis unit can analyze the frequency of crying and determine the stress state of the family members. For example, if there is a lot of crying, the analysis unit can determine that the family members are feeling stressed. In this way, the living conditions of the family members can be determined in detail by analyzing the voice data. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the analysis unit can use a generative AI model that takes voice data as input and outputs whether the person is living a healthy life, feeling lonely, or at risk of illness, depression, or mental illness.

[0035] The notification unit can notify the relevant person based on the analysis result. For example, the notification unit can send a notification to the relative that "the person looks lonely" based on the analysis result. For example, the notification unit can send a notification to the relative that "the person is living well" based on the analysis result. The notification unit can also send a notification to the relative that "there are signs of illness" based on the analysis result. Furthermore, the notification unit can also send a notification to the relative that "there is a risk of mental abnormality" based on the analysis result. This allows for a quick response by notifying the relevant person based on the analysis result. Part or all of the above-mentioned processing in the notification unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the notification unit can generate the notification content using a generation AI model that inputs the analysis result and outputs the notification content.

[0036] The notification unit can specifically generate notification contents. The notification unit can generate specific notification contents based on, for example, the analysis results and send them to the relatives. For example, the notification unit can generate notification contents such as "Looks lonely" based on the analysis results and send them to the relatives. The notification unit can also generate notification contents such as "Living well" based on the analysis results and send them to the relatives. The notification unit can also generate notification contents such as "There are signs of illness" based on the analysis results and send them to the relatives. The notification unit can also generate notification contents such as "There is a risk of mental abnormality" based on the analysis results and send them to the relatives. By generating specific notification contents, the relevant parties can accurately grasp the situation. Part or all of the above-mentioned processing in the notification unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the notification unit can generate notification contents using a generation AI model that inputs the analysis results and outputs the notification contents.

[0037] The collection unit can also collect data from a motion sensor, a temperature sensor, and the like. The collection unit can, for example, detect the movement of family members using a motion sensor and collect the data. For example, the collection unit can detect how family members are moving in the house using a motion sensor and collect the data. The collection unit can also collect temperature data in the house using a temperature sensor. For example, the collection unit can detect whether the temperature in the house is appropriate using a temperature sensor and collect the data. Furthermore, the collection unit can also collect humidity data in the house using a humidity sensor. For example, the collection unit can detect whether the humidity in the house is appropriate using a humidity sensor and collect the data. In this way, by collecting data from other sensors, the living conditions of the family can be grasped from a more multifaceted perspective. Part or all of the above-mentioned processing in the collection unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the collection unit can input data acquired by a motion sensor or a temperature sensor into the generation AI and have the generation AI execute analysis of the data.

[0038] The collection unit can select an appropriate collection method by referring to past voice data of family members when collecting voice data. The collection unit, for example, can refer to past voice data of family members when collecting voice data. For example, the collection unit can confirm from the past voice data that many voices are generated in a specific time period and concentrate collection on that time period. The collection unit can also analyze the past voice data and preferentially collect voices related to a specific event (e.g., during meals, while watching TV). Furthermore, the collection unit can identify a specific emotional state (e.g., a time period when there is a lot of laughter) from the past voice data and concentrate collection on that time period. This makes it possible to select an optimal collection method by referring to the past voice data. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the collection unit can input past voice data to the generation AI and cause the generation AI to select an optimal collection method.

[0039] The collection unit can perform filtering based on the behavioral patterns of the family when collecting the voice data. For example, the collection unit analyzes the behavioral patterns of the family when collecting the voice data. For example, the collection unit analyzes the life patterns of the family and filters the voice data to be collected in a specific time period. The collection unit can also preferentially collect voice data related to a specific event (e.g., during meals, before going to bed) based on the life patterns of the family. Furthermore, the collection unit can filter voice data related to a specific emotional state (e.g., a time period when the family is relaxing) in consideration of the life patterns of the family. This makes it possible to preferentially collect important data by filtering based on the life patterns. A part or all of the above-mentioned processing in the collection unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the collection unit can input the behavioral patterns of the family to the generation AI and cause the generation AI to execute filtering.

[0040] The collection unit can preferentially collect highly relevant data based on the geographical location information of the family members when collecting the voice data. The collection unit, for example, considers the geographical location information of the family members when collecting the voice data. For example, when the family members are in a specific place (e.g., living room), the collection unit preferentially collects voice data generated at that place. In addition, when the family members are in a specific place (e.g., kitchen), the collection unit can preferentially collect voice data generated at that place. Furthermore, when the family members are in a specific place (e.g., bedroom), the collection unit can preferentially collect voice data generated at that place. In this way, by considering the geographical location information, highly relevant data can be preferentially collected. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit can input the geographical location information to the generation AI and cause the generation AI to execute collection of highly relevant data.

[0041] The collection unit can analyze the social media usage of the family members when collecting the voice data, and collect related voice data. The collection unit, for example, analyzes the social media activities of the family members when collecting the voice data. For example, the collection unit preferentially collects voice data related to a specific event (e.g., birthday party) from the social media activities of the family members. The collection unit can also analyze the social media activities of the family members and preferentially collect voice data related to a specific emotional state (e.g., joy, sadness). Furthermore, the collection unit can preferentially collect voice data related to a specific time period from the social media activities of the family members. In this way, related voice data can be collected by analyzing the social media activities. A part or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit can input the social media activities to the generation AI and cause the generation AI to collect related voice data.

[0042] The analysis unit can adjust the level of detail of the analysis based on the importance of the voice data during analysis. The analysis unit, for example, evaluates the importance of the voice data. For example, the analysis unit can evaluate the importance based on the contents of the voice data and the state of the speaker. Next, the analysis unit adjusts the level of detail of the analysis based on the importance. For example, the analysis unit analyzes voice data with high importance (e.g., crying, coughing) in detail to grasp the specific situation. In addition, the analysis unit can briefly analyze voice data with low importance (e.g., daily conversation) to grasp the overall trend. Furthermore, the analysis unit can moderately analyze voice data with medium importance (e.g., laughter) to grasp the situation. This allows efficient analysis by adjusting the level of detail based on the importance. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input voice data to the generation AI and cause the generation AI to evaluate the importance and adjust the level of detail.

[0043] The analysis unit can apply different analysis algorithms depending on the category of the voice data during analysis. The analysis unit, for example, classifies the category of the voice data. For example, the analysis unit can classify the voice data into categories such as talking, laughing, and crying. Next, the analysis unit applies different analysis algorithms depending on the category. For example, the analysis unit applies a voice recognition algorithm to the voice data of the talking voice and analyzes the content. In addition, the analysis unit can apply an emotion analysis algorithm to the voice data of the laughing voice and grasp the emotional state. Furthermore, the analysis unit can apply a stress analysis algorithm to the voice data of the crying voice and grasp the stress level. This enables highly accurate analysis by applying an analysis algorithm according to the category. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the analysis unit can input the voice data to the generation AI and cause the generation AI to execute the classification of the category and the application of the analysis algorithm.

[0044] During analysis, the analysis unit can determine the priority of analysis based on the collection time of the voice data. The analysis unit, for example, evaluates the collection time of the voice data. For example, the analysis unit can determine the priority based on the collection time of the voice data. Next, the analysis unit determines the priority of analysis based on the collection time. For example, the analysis unit preferentially analyzes recently collected voice data to grasp the latest situation. In addition, the analysis unit can preferentially analyze voice data related to a specific event (e.g., birthday, anniversary). Furthermore, the analysis unit can refer to past voice data and analyze it to grasp a long-term trend. Thereby, by determining the priority based on the collection time, the latest situation can be quickly grasped. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the analysis unit can input the collection time of the voice data to the generation AI and cause the generation AI to execute the determination of the priority.

[0045] The analysis unit can adjust the order of analysis based on the relevance of the voice data during analysis. The analysis unit, for example, evaluates the relevance of the voice data. For example, the analysis unit can evaluate the relevance based on the content of the voice data and the state of the speaker. Next, the analysis unit adjusts the order of analysis based on the relevance. For example, the analysis unit prioritizes analysis of important voice data (e.g., crying, coughing) to quickly grasp the situation. In addition, the analysis unit prioritizes analysis of highly relevant voice data (e.g., talking, laughter) to grasp the overall situation. Furthermore, the analysis unit can analyze less relevant voice data (e.g., background sound) later. As a result, important data can be analyzed preferentially by adjusting the order based on the relevance. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input the relevance of the voice data to the generation AI and cause the generation AI to adjust the order of analysis.

[0046] The notification unit can adjust the level of detail of the notification based on the importance of the analysis result at the time of notification. The notification unit, for example, evaluates the importance of the analysis result. For example, the notification unit can evaluate the importance based on the content and urgency of the analysis result. Next, the notification unit adjusts the level of detail of the notification based on the importance. For example, the notification unit notifies in detail an analysis result with high importance (e.g., signs of illness) and proposes specific measures. In addition, the notification unit can briefly notify an analysis result with low importance (e.g., daily situation) to grasp the overall trend. Furthermore, the notification unit can appropriately notify an analysis result with medium importance (e.g., looking lonely) to grasp the situation. This enables efficient notification by adjusting the level of detail based on the importance. Part or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the analysis result to the generation AI and cause the generation AI to evaluate the importance and adjust the level of detail.

[0047] The notification unit can apply different notification algorithms depending on the category of the analysis result at the time of notification. The notification unit, for example, classifies the categories of the analysis result. For example, the notification unit can classify the analysis result into categories such as health condition, emergency call, and emotional state. Next, the notification unit applies different notification algorithms depending on the category. For example, the notification unit applies an emergency notification algorithm to the analysis result regarding the signs of illness and notifies promptly. In addition, the notification unit can apply a periodic notification algorithm to the analysis result regarding daily situations and notify periodically. Furthermore, the notification unit can apply an emotion notification algorithm to the analysis result regarding the emotional state and notify at an appropriate time. Thereby, by applying a notification algorithm according to the category, it is possible to notify at an appropriate time. Part or all of the above-mentioned processing in the notification unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the notification unit can input the analysis result to the generation AI and cause the generation AI to execute the classification of the category and the application of the notification algorithm.

[0048] The notification unit can determine the priority of the notification based on the collection time of the analysis result at the time of notification. The notification unit, for example, evaluates the collection time of the analysis result. For example, the notification unit can determine the priority based on the collection time of the analysis result. Next, the notification unit determines the priority of the notification based on the collection time. For example, the notification unit prioritizes notification of the analysis result collected recently to grasp the latest situation. In addition, the notification unit can prioritize notification of the analysis result related to a specific event (e.g., birthday, anniversary). Furthermore, the notification unit can refer to past analysis results and notify to grasp long-term trends. Thereby, by determining the priority based on the collection time, the latest situation can be quickly grasped. Part or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the collection time of the analysis result to the generation AI and cause the generation AI to execute the determination of the priority.

[0049] The notification unit can adjust the order of notifications based on the relevance of the analysis results at the time of notification. The notification unit, for example, evaluates the relevance of the analysis results. For example, the notification unit can evaluate the relevance based on the contents of the analysis results and the state of the speaker. Next, the notification unit adjusts the order of notifications based on the relevance. For example, the notification unit prioritizes notification of important analysis results (e.g., signs of illness) to enable a quick response. In addition, the notification unit prioritizes notification of highly relevant analysis results (e.g., looking lonely) to enable the situation to be grasped. Furthermore, the notification unit can postpone notification of less relevant analysis results (e.g., daily situations). As a result, important data can be prioritized by adjusting the order based on the relevance. Part or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the relevance of the analysis results to the generation AI and cause the generation AI to adjust the order of notifications.

[0050] The collection unit can select the optimal collection method by referring to the past data of the family when collecting the sensor data. The collection unit, for example, refers to the past data of the family when collecting the sensor data. For example, the collection unit confirms from the past sensor data that a lot of data occurs in a specific time period, and concentrates collection on that time period. The collection unit can also analyze the past sensor data and preferentially collect data related to a specific event (e.g., during exercise, during rest). Furthermore, the collection unit can identify a specific emotional state (e.g., a time period when the person is relaxed) from the past sensor data, and concentrate collection on that time period. This makes it possible to select the optimal collection method by referring to the past data. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit can input past data to the generation AI and cause the generation AI to select the optimal collection method.

[0051] The collection unit can preferentially collect highly relevant data in consideration of the geographical location information of the family when collecting the sensor data. The collection unit, for example, considers the geographical location information of the family when collecting the sensor data. For example, when the family is in a specific place (e.g., living room), the collection unit preferentially collects sensor data generated in that place. Furthermore, when the family is in a specific place (e.g., kitchen), the collection unit can preferentially collect sensor data generated in that place. Furthermore, when the family is in a specific place (e.g., bedroom), the collection unit can preferentially collect sensor data generated in that place. In this way, by considering the geographical location information, highly relevant data can be preferentially collected. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit can input the geographical location information to the generation AI and cause the generation AI to collect highly relevant data.

[0052] The analysis unit can adjust the level of detail of the analysis based on the importance of the voice data during analysis. The analysis unit, for example, evaluates the importance of the voice data. For example, the analysis unit can evaluate the importance based on the contents of the voice data and the state of the speaker. Next, the analysis unit adjusts the level of detail of the analysis based on the importance. For example, the analysis unit analyzes voice data with high importance (e.g., crying, coughing) in detail to grasp the specific situation. In addition, the analysis unit can briefly analyze voice data with low importance (e.g., daily conversation) to grasp the overall trend. Furthermore, the analysis unit can moderately analyze voice data with medium importance (e.g., laughter) to grasp the situation. This allows efficient analysis by adjusting the level of detail based on the importance. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input voice data to the generation AI and cause the generation AI to evaluate the importance and adjust the level of detail.

[0053] During analysis, the analysis unit can determine the priority of analysis based on the collection time of the voice data. The analysis unit, for example, evaluates the collection time of the voice data. For example, the analysis unit can determine the priority based on the collection time of the voice data. Next, the analysis unit determines the priority of analysis based on the collection time. For example, the analysis unit preferentially analyzes recently collected voice data to grasp the latest situation. In addition, the analysis unit can preferentially analyze voice data related to a specific event (e.g., birthday, anniversary). Furthermore, the analysis unit can refer to past voice data and analyze it to grasp a long-term trend. Thereby, by determining the priority based on the collection time, the latest situation can be quickly grasped. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the analysis unit can input the collection time of the voice data to the generation AI and cause the generation AI to execute the determination of the priority.

[0054] The notification unit can adjust the level of detail of the notification based on the importance of the analysis result at the time of notification. The notification unit, for example, evaluates the importance of the analysis result. For example, the notification unit can evaluate the importance based on the content and urgency of the analysis result. Next, the notification unit adjusts the level of detail of the notification based on the importance. For example, the notification unit notifies in detail an analysis result with high importance (e.g., signs of illness) and proposes specific measures. In addition, the notification unit can briefly notify an analysis result with low importance (e.g., daily situation) to grasp the overall trend. Furthermore, the notification unit can appropriately notify an analysis result with medium importance (e.g., looking lonely) to grasp the situation. This enables efficient notification by adjusting the level of detail based on the importance. Part or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the analysis result to the generation AI and cause the generation AI to evaluate the importance and adjust the level of detail.

[0055] The notification unit can determine the priority of the notification based on the collection time of the analysis result at the time of notification. The notification unit, for example, evaluates the collection time of the analysis result. For example, the notification unit can determine the priority based on the collection time of the analysis result. Next, the notification unit determines the priority of the notification based on the collection time. For example, the notification unit prioritizes notification of the analysis result collected recently to grasp the latest situation. In addition, the notification unit can prioritize notification of the analysis result related to a specific event (e.g., birthday, anniversary). Furthermore, the notification unit can refer to past analysis results and notify to grasp long-term trends. Thereby, by determining the priority based on the collection time, the latest situation can be quickly grasped. Part or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the collection time of the analysis result to the generation AI and cause the generation AI to execute the determination of the priority.

[0056] The notification unit can adjust the order of notifications based on the relevance of the analysis results at the time of notification. The notification unit, for example, evaluates the relevance of the analysis results. For example, the notification unit can evaluate the relevance based on the contents of the analysis results and the state of the speaker. Next, the notification unit adjusts the order of notifications based on the relevance. For example, the notification unit prioritizes notification of important analysis results (e.g., signs of illness) to enable a quick response. In addition, the notification unit prioritizes notification of highly relevant analysis results (e.g., looking lonely) to enable the situation to be grasped. Furthermore, the notification unit can postpone notification of less relevant analysis results (e.g., daily situations). As a result, important data can be prioritized by adjusting the order based on the relevance. Part or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the relevance of the analysis results to the generation AI and cause the generation AI to adjust the order of notifications.

[0057] The collection unit can select the optimal collection method by referring to the past data of the family when collecting the sensor data. The collection unit, for example, refers to the past data of the family when collecting the sensor data. For example, the collection unit confirms from the past sensor data that a lot of data occurs in a specific time period, and concentrates collection on that time period. The collection unit can also analyze the past sensor data and preferentially collect data related to a specific event (e.g., during exercise, during rest). Furthermore, the collection unit can identify a specific emotional state (e.g., a time period when the person is relaxed) from the past sensor data, and concentrate collection on that time period. This makes it possible to select the optimal collection method by referring to the past data. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit can input past data to the generation AI and cause the generation AI to select the optimal collection method.

[0058] The collection unit can preferentially collect highly relevant data in consideration of the geographical location information of the family when collecting the sensor data. The collection unit, for example, considers the geographical location information of the family when collecting the sensor data. For example, when the family is in a specific place (e.g., living room), the collection unit preferentially collects sensor data generated in that place. Furthermore, when the family is in a specific place (e.g., kitchen), the collection unit can preferentially collect sensor data generated in that place. Furthermore, when the family is in a specific place (e.g., bedroom), the collection unit can preferentially collect sensor data generated in that place. In this way, by considering the geographical location information, highly relevant data can be preferentially collected. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit can input the geographical location information to the generation AI and cause the generation AI to collect highly relevant data.

[0059] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0060] The collection unit can learn the life rhythms of family members and automatically adjust the optimal collection timing. For example, the collection unit can start collecting voice data when family members usually wake up and stop collecting when they go to bed. The collection unit can also focus collection on times when family members perform specific activities (e.g., eating, exercising). Furthermore, the collection unit can adjust the collection timing on special days such as weekends and holidays based on the family members' life rhythms. This allows more accurate data to be collected by optimizing the collection timing based on the life rhythms.

[0061] The analysis unit can visually display the analysis results of the voice data. For example, the analysis unit can display the emotional state of family members in graphs and charts to make it easier to understand visually. The analysis unit can also display the tone and frequency of the voice data in different colors to make it possible to grasp changes in emotions at a glance. Furthermore, the analysis unit can display the analysis results in a timeline format to visually track changes in emotions over time. This allows the analysis results to be more intuitively understood through visual displays.

[0062] The notification unit can suggest specific action plans for family members based on the analysis results. For example, if a family member looks lonely, the notification unit can suggest specific actions such as "calling" or "visiting" relatives. In addition, if a family member shows signs of illness, the notification unit can send a notification encouraging the family member to visit a medical institution. Furthermore, if a family member looks relaxed, the notification unit can suggest activities to maintain relaxation (e.g., walking, reading). This makes it possible to respond quickly and appropriately by suggesting specific action plans.

[0063] The collection unit can collect health data (e.g., heart rate, blood pressure) of family members and use it to understand their living conditions. For example, the collection unit can collect heart rate data of family members using a heart rate sensor and evaluate their stress levels. The collection unit can also collect blood pressure data of family members using a blood pressure sensor and monitor their health conditions. Furthermore, the collection unit can collect sleep data of family members using a sleep sensor and evaluate their sleep quality. This makes it possible to understand the living conditions more comprehensively by collecting health data.

[0064] The analysis unit can analyze the family's living conditions in combination with other data sources (e.g., weather information, calendar information). For example, the analysis unit can refer to weather information and analyze whether the family spends more time indoors on bad weather days. The analysis unit can also refer to calendar information and analyze changes in emotions related to specific events (e.g., birthdays, anniversaries). Furthermore, the analysis unit can refer to traffic information and analyze the impact of fluctuations in commuting time on the family's stress level. This allows for more accurate analysis by combining with other data sources.

[0065] The process flow of the first embodiment will be briefly described below.

[0066] Step 1: The collection unit collects voice data. The voice data includes talking, laughing, crying, coughing, etc. The collection unit collects the voice data in real time using voice sensors installed in the house. The collection unit can also collect data from other sensors such as motion sensors and temperature sensors. Step 2: The analysis unit uses the generation AI to analyze the voice data collected by the collection unit. The analysis is carried out using voice recognition technology and emotion analysis algorithms. For example, the tone and frequency of speech, the presence or absence of laughter, the frequency of crying, etc. are analyzed to determine whether the person is living a healthy life, seems lonely, or is at risk of illness, depression, or mental disorder. Step 3: The notification unit issues a notification based on the analysis results obtained by the analysis unit. The notification is given by email notification, app notification, voice notification, or other methods. For example, based on the analysis results, a notification such as "You look lonely" can be sent to relatives. The notification unit can also generate specific notification content based on the analysis results and send it to relatives.

[0067] (Example 2) The living situation grasping system according to the embodiment of the present invention is a system for grasping the living situation of family members living apart. In this system, a voice sensor installed in a house collects voice data, and a generation AI analyzes the voice data to judge the living situation of the family members. For example, it judges whether the family members are living well, seem lonely, or are at risk of illness, depression, or mental abnormality. Based on the judgment result, a notification is sent to the relevant person (basically a relative). This notification is used as an opportunity to provide some help in advance or as a means of communication. For example, the living situation grasping system installs a voice sensor in a house. This voice sensor collects voice data in the house in real time. For example, it collects voice data such as talking, laughter, crying, and coughing. This voice data is sent to the generation AI. Next, the generation AI analyzes the voice data. The generation AI analyzes the voice data and judges the living situation of the family members. For example, it judges whether the family members are living well, seem lonely, or are at risk of illness, depression, or mental abnormality by analyzing the tone and frequency of the talking, the presence or absence of laughter, and the frequency of crying. Based on the judgment result, a notification is sent to the relevant person. For example, if the generation AI judges that the person looks lonely, a notification that "the person looks lonely" is sent to the relatives. This notification is used as an opportunity for the relatives to provide some help in advance or as a means of communication. This system makes it possible to grasp the living conditions of family members living apart in real time, and to provide necessary help and communication quickly. For example, if a parent lives alone, the children can grasp the living conditions of the parent and provide necessary help quickly. Also, if a child lives alone, the parent can grasp the living conditions of the child and provide necessary help quickly. In this way, the living conditions grasping system can grasp the living conditions of family members living apart in real time, and provide necessary help and communication quickly.

[0068] The living situation grasping system according to the embodiment includes a collection unit, an analysis unit, and a notification unit. The collection unit collects voice data. The voice data includes, for example, talking, laughing, crying, coughing, and the like, but is not limited to such examples. The collection unit collects voice data such as talking and laughing in real time, for example, using a voice sensor installed in the house. The collection unit can also collect data from other sensors (such as a motion sensor and a temperature sensor). For example, the collection unit can detect the movement of family members using a motion sensor and collect the data. The collection unit can also collect temperature data in the house using a temperature sensor. The analysis unit analyzes the voice data collected by the collection unit using a generation AI. The analysis is performed using, for example, a voice recognition technology or an emotion analysis algorithm. For example, the analysis unit analyzes the tone and frequency of the talking, the presence or absence of laughter, the frequency of crying, and the like, and judges whether the family members are living energetically, seem lonely, or are at risk of illness, depression, or mental abnormality. The analysis unit can analyze the voice data using the generation AI to judge the living situation of the family members. For example, the analysis unit can perform analysis using a generation AI model that inputs voice data and outputs whether the person is living well, seems lonely, or is at risk of illness, depression, or mental abnormality. The notification unit issues a notification based on the analysis result obtained by the analysis unit. The notification is performed, for example, by email notification, app notification, voice notification, or other methods. For example, the notification unit can send a notification to a relative saying "I seem lonely" based on the analysis result. The notification unit can also specifically generate notification content. For example, the notification unit can generate specific notification content based on the analysis result and send it to the relative. As a result, the living situation grasping system according to the embodiment can grasp the living situation of family members living apart in real time and can quickly provide necessary help and communication. A part or all of the above-mentioned processing in the notification unit may be performed, for example, using AI, or may be performed without using AI. For example, the notification unit can generate notification content using an AI model that inputs the analysis result and outputs notification content.

[0069] The collection unit collects voice data. The voice data includes, for example, talking, laughing, crying, coughing, and the like, but is not limited to such examples. For example, the collection unit uses a voice sensor installed in the house to collect voice data such as talking and laughing in real time. The voice sensor is installed in each room of the house and can capture the living sounds of the family with high sensitivity. This makes it possible to grasp the living conditions of the family in detail. The collection unit can also collect data from other sensors (motion sensors, temperature sensors, etc.). For example, the collection unit can detect the movement of the family using a motion sensor and collect the data. The motion sensor is installed in the main passages and rooms in the house and monitors the movement and activities of the family in real time. This makes it possible to grasp which room the family is in and what activities they are doing. The collection unit can also collect temperature data in the house using a temperature sensor. The temperature sensor is installed in each room of the house and monitors the temperature change in each room in real time. This makes it possible to confirm whether the family is living in a comfortable environment. Furthermore, the collection unit can centrally manage these data and cooperate with other systems and departments as necessary. For example, the collected data is stored in a cloud server and made accessible to the analysis and notification units. In addition, by adjusting the frequency and accuracy of data collection, flexible responses to specific situations and conditions are possible. This allows the collection unit to collect data efficiently and effectively, improving the performance of the entire system.

[0070] The analysis unit uses the generation AI to analyze the voice data collected by the collection unit. The analysis is performed, for example, using voice recognition technology and emotion analysis algorithms. Specifically, the content of the speech is converted into text using voice recognition technology, and emotion analysis is performed based on the text data. The emotion analysis algorithm analyzes the tone and frequency of the speech, the presence or absence of laughter, the frequency of crying, etc., and judges whether the family is living happily, seems lonely, or is at risk of illness, depression, or mental abnormality. The analysis unit can analyze the voice data using the generation AI and judge the family's living situation. For example, the analysis unit can perform analysis using a generation AI model that takes voice data as input and outputs whether the family is living happily, seems lonely, or is at risk of illness, depression, or mental abnormality. The generation AI model has learned a large amount of voice data and its analysis results, and can perform highly accurate analysis. In addition, the analysis unit can also evaluate long-term changes in living conditions by utilizing past data and statistical information. For example, based on past voice data, it analyzes changes in family members' emotions and changes in their living patterns to predict future risks. In addition, the analysis unit can use anomaly detection algorithms to detect unusual patterns or abnormal data and issue early warnings, allowing the analysis unit to not only grasp the situation in real time, but also handle long-term risk management and anomaly detection, improving the reliability and safety of the entire system.

[0071] The notification unit issues a notification based on the analysis result obtained by the analysis unit. The notification is, for example, by email notification, app notification, voice notification, or the like. Specifically, based on the analysis result, a notification such as "I look lonely" can be sent to the relative. The notification unit can generate specific notification content based on the analysis result and send it to the relative. For example, if the analysis result is determined to be "I look lonely," the notification unit generates specific notification content such as "I look lonely recently. Please call me." and sends it to the relative. The notification unit can also customize the notification content. For example, the details and frequency of the notification content can be adjusted according to the relative's wishes. Furthermore, the notification unit can reliably transmit information using multiple communication means. For example, important information can be reliably delivered by using not only smartphone notifications but also voice calls, SMS, emails, and the like. This allows the notification unit to quickly and reliably provide action instructions to the user and minimize the risk of disaster. Furthermore, the notification unit can collect user feedback and continuously improve the accuracy and effectiveness of the notification content. For example, the notification content is reviewed and improved based on feedback from the relative who received the notification. This enables the notification unit to always provide highly accurate notifications based on the latest information, and to assist in prompt and appropriate responses.

[0072] The collection unit can collect voice data such as talking, laughter, crying, coughing, and other voice data. The collection unit collects voice data such as talking and laughter in real time, for example, using a voice sensor installed in the house. For example, the collection unit collects the tone and frequency of the talking voice to grasp the living conditions of the family. The collection unit can also collect the presence or absence of laughter to grasp the emotional state of the family. Furthermore, the collection unit can collect the frequency of crying voice to grasp the stress state of the family. For example, the collection unit can determine that the family is living a healthy life when the tone of the talking voice is high. Furthermore, the collection unit can determine that the family is relaxed when there is a lot of laughter. Furthermore, the collection unit can determine that the family is feeling stressed when there is a lot of crying. In this way, by collecting various voice data, the living conditions of the family can be grasped in detail. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using AI, or may be performed without using AI. For example, the collection unit can input the voice data acquired by the voice sensor to the generation AI and cause the generation AI to execute analysis of the voice data.

[0073] The analysis unit can analyze the voice data and determine whether the family members are living well, feeling lonely, or at risk of illness, depression, or mental abnormality. The analysis unit, for example, uses voice recognition technology to analyze the tone and frequency of the speech. For example, if the tone of the speech is high, the analysis unit can determine that the family members are living well. Also, if the tone of the speech is low, the analysis unit can determine that the family members feel lonely. Also, the analysis unit can analyze the presence or absence of laughter using an emotion analysis algorithm. For example, if there is a lot of laughter, the analysis unit can determine that the family members are relaxed. Also, the analysis unit can analyze the frequency of crying and determine the stress state of the family members. For example, if there is a lot of crying, the analysis unit can determine that the family members are feeling stressed. In this way, the living conditions of the family members can be determined in detail by analyzing the voice data. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the analysis unit can use a generative AI model that takes voice data as input and outputs whether the person is living a healthy life, feeling lonely, or at risk of illness, depression, or mental illness.

[0074] The notification unit can notify the relevant person based on the analysis result. For example, the notification unit can send a notification to the relative that "the person looks lonely" based on the analysis result. For example, the notification unit can send a notification to the relative that "the person is living well" based on the analysis result. The notification unit can also send a notification to the relative that "there are signs of illness" based on the analysis result. Furthermore, the notification unit can also send a notification to the relative that "there is a risk of mental abnormality" based on the analysis result. This allows for a quick response by notifying the relevant person based on the analysis result. Part or all of the above-mentioned processing in the notification unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the notification unit can generate the notification content using a generation AI model that inputs the analysis result and outputs the notification content.

[0075] The notification unit can specifically generate notification contents. The notification unit can generate specific notification contents based on, for example, the analysis results and send them to the relatives. For example, the notification unit can generate notification contents such as "Looks lonely" based on the analysis results and send them to the relatives. The notification unit can also generate notification contents such as "Living well" based on the analysis results and send them to the relatives. The notification unit can also generate notification contents such as "There are signs of illness" based on the analysis results and send them to the relatives. The notification unit can also generate notification contents such as "There is a risk of mental abnormality" based on the analysis results and send them to the relatives. By generating specific notification contents, the relevant parties can accurately grasp the situation. Part or all of the above-mentioned processing in the notification unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the notification unit can generate notification contents using a generation AI model that inputs the analysis results and outputs the notification contents.

[0076] The collection unit can also collect data from a motion sensor, a temperature sensor, and the like. The collection unit can, for example, detect the movement of family members using a motion sensor and collect the data. For example, the collection unit can detect how family members are moving in the house using a motion sensor and collect the data. The collection unit can also collect temperature data in the house using a temperature sensor. For example, the collection unit can detect whether the temperature in the house is appropriate using a temperature sensor and collect the data. Furthermore, the collection unit can also collect humidity data in the house using a humidity sensor. For example, the collection unit can detect whether the humidity in the house is appropriate using a humidity sensor and collect the data. In this way, by collecting data from other sensors, the living conditions of the family can be grasped from a more multifaceted perspective. Part or all of the above-mentioned processing in the collection unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the collection unit can input data acquired by a motion sensor or a temperature sensor into the generation AI and have the generation AI execute analysis of the data.

[0077] The collection unit can estimate the user's emotion and adjust the collection timing of the voice data based on the estimated user's emotion. The collection unit, for example, estimates the user's emotion when collecting the voice data. For example, the collection unit can analyze the voice data and estimate whether the user is relaxed or stressed. Next, the collection unit adjusts the collection timing of the voice data based on the estimated user's emotion. For example, when the user is relaxed, the collection unit sets the collection frequency of the voice data low and collects only necessary data. Also, when the user is stressed, the collection unit can set the collection frequency of the voice data high and collect detailed data. Furthermore, when the user is in a hurry, the collection unit can collect a lot of voice data in a short time and perform analysis quickly. As a result, more appropriate data can be collected by adjusting the collection timing based on the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit may input voice data to the generation AI and cause the generation AI to estimate the user's emotions.

[0078] The collection unit can select an appropriate collection method by referring to past voice data of family members when collecting voice data. The collection unit, for example, can refer to past voice data of family members when collecting voice data. For example, the collection unit can confirm from the past voice data that many voices are generated in a specific time period and concentrate collection on that time period. The collection unit can also analyze the past voice data and preferentially collect voices related to a specific event (e.g., during meals, while watching TV). Furthermore, the collection unit can identify a specific emotional state (e.g., a time period when there is a lot of laughter) from the past voice data and concentrate collection on that time period. This makes it possible to select an optimal collection method by referring to the past voice data. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the collection unit can input past voice data to the generation AI and cause the generation AI to select an optimal collection method.

[0079] The collection unit can perform filtering based on the behavioral patterns of the family when collecting the voice data. For example, the collection unit analyzes the behavioral patterns of the family when collecting the voice data. For example, the collection unit analyzes the life patterns of the family and filters the voice data to be collected in a specific time period. The collection unit can also preferentially collect voice data related to a specific event (e.g., during meals, before going to bed) based on the life patterns of the family. Furthermore, the collection unit can filter voice data related to a specific emotional state (e.g., a time period when the family is relaxing) in consideration of the life patterns of the family. This makes it possible to preferentially collect important data by filtering based on the life patterns. A part or all of the above-mentioned processing in the collection unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the collection unit can input the behavioral patterns of the family to the generation AI and cause the generation AI to execute filtering.

[0080] The collection unit can estimate the user's emotion and determine the priority of the voice data to be collected based on the estimated user's emotion. The collection unit, for example, estimates the user's emotion when collecting the voice data. For example, the collection unit can analyze the voice data and estimate whether the user is relaxed or stressed. Next, the collection unit determines the priority of the voice data to be collected based on the estimated user's emotion. For example, the collection unit preferentially collects laughter and talking when the user is relaxed. Also, the collection unit can preferentially collect voice data such as crying and coughing when the user is stressed. Furthermore, the collection unit can preferentially collect voice data that can be collected in a short time when the user is in a hurry. In this way, important data can be preferentially collected by determining the priority based on the emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the collection unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the collection unit can input voice data to the generation AI and have the generation AI estimate the user's emotions.

[0081] The collection unit can preferentially collect highly relevant data based on the geographical location information of the family members when collecting the voice data. The collection unit, for example, considers the geographical location information of the family members when collecting the voice data. For example, when the family members are in a specific place (e.g., living room), the collection unit preferentially collects voice data generated at that place. In addition, when the family members are in a specific place (e.g., kitchen), the collection unit can preferentially collect voice data generated at that place. Furthermore, when the family members are in a specific place (e.g., bedroom), the collection unit can preferentially collect voice data generated at that place. In this way, by considering the geographical location information, highly relevant data can be preferentially collected. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit can input the geographical location information to the generation AI and cause the generation AI to execute collection of highly relevant data.

[0082] The collection unit can analyze the social media usage of the family members when collecting the voice data, and collect related voice data. The collection unit, for example, analyzes the social media activities of the family members when collecting the voice data. For example, the collection unit preferentially collects voice data related to a specific event (e.g., birthday party) from the social media activities of the family members. The collection unit can also analyze the social media activities of the family members and preferentially collect voice data related to a specific emotional state (e.g., joy, sadness). Furthermore, the collection unit can preferentially collect voice data related to a specific time period from the social media activities of the family members. In this way, related voice data can be collected by analyzing the social media activities. A part or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit can input the social media activities to the generation AI and cause the generation AI to collect related voice data.

[0083] The analysis unit can estimate the user's emotion and adjust the expression method of the analysis based on the estimated user's emotion. The analysis unit estimates the user's emotion, for example, when analyzing voice data. For example, the analysis unit can analyze the voice data and estimate whether the user is relaxed or stressed. Next, the analysis unit adjusts the expression method of the analysis based on the estimated user's emotion. For example, when the user is relaxed, the analysis unit displays the analysis result in detail and provides it in an easy-to-understand format. In addition, when the user is stressed, the analysis unit can display the analysis result concisely and provide it in a format that can be quickly understood. Furthermore, when the user is in a hurry, the analysis unit can display only the main points of the analysis result and provide it in a format that can be quickly understood. Thereby, by adjusting the expression method based on the emotion, the analysis result can be provided in a more easily understandable manner. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the analysis unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the analysis unit may input voice data to the generation AI and cause the generation AI to estimate the user's emotions.

[0084] The analysis unit can adjust the level of detail of the analysis based on the importance of the voice data during analysis. The analysis unit, for example, evaluates the importance of the voice data. For example, the analysis unit can evaluate the importance based on the contents of the voice data and the state of the speaker. Next, the analysis unit adjusts the level of detail of the analysis based on the importance. For example, the analysis unit analyzes voice data with high importance (e.g., crying, coughing) in detail to grasp the specific situation. In addition, the analysis unit can briefly analyze voice data with low importance (e.g., daily conversation) to grasp the overall trend. Furthermore, the analysis unit can moderately analyze voice data with medium importance (e.g., laughter) to grasp the situation. This allows efficient analysis by adjusting the level of detail based on the importance. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input voice data to the generation AI and cause the generation AI to evaluate the importance and adjust the level of detail.

[0085] The analysis unit can apply different analysis algorithms depending on the category of the voice data during analysis. The analysis unit, for example, classifies the category of the voice data. For example, the analysis unit can classify the voice data into categories such as talking, laughing, and crying. Next, the analysis unit applies different analysis algorithms depending on the category. For example, the analysis unit applies a voice recognition algorithm to the voice data of the talking voice and analyzes the content. In addition, the analysis unit can apply an emotion analysis algorithm to the voice data of the laughing voice and grasp the emotional state. Furthermore, the analysis unit can apply a stress analysis algorithm to the voice data of the crying voice and grasp the stress level. This enables highly accurate analysis by applying an analysis algorithm according to the category. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the analysis unit can input the voice data to the generation AI and cause the generation AI to execute the classification of the category and the application of the analysis algorithm.

[0086] The analysis unit can estimate the user's emotion and adjust the length of the analysis based on the estimated user's emotion. The analysis unit, for example, estimates the user's emotion when analyzing the voice data. For example, the analysis unit can analyze the voice data and estimate whether the user is relaxed or stressed. Next, the analysis unit adjusts the length of the analysis based on the estimated user's emotion. For example, the analysis unit can provide a detailed analysis result and generate a longer report when the user is relaxed. Also, the analysis unit can provide a concise analysis result and generate a shorter report when the user is stressed. Furthermore, the analysis unit can provide a short analysis result including only the main points when the user is in a hurry. Thereby, by adjusting the length of the analysis based on the emotion, an analysis result suitable for the user can be provided. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input voice data to the generation AI and have the generation AI estimate the user's emotions and adjust the length of the analysis.

[0087] During analysis, the analysis unit can determine the priority of analysis based on the collection time of the voice data. The analysis unit, for example, evaluates the collection time of the voice data. For example, the analysis unit can determine the priority based on the collection time of the voice data. Next, the analysis unit determines the priority of analysis based on the collection time. For example, the analysis unit preferentially analyzes recently collected voice data to grasp the latest situation. In addition, the analysis unit can preferentially analyze voice data related to a specific event (e.g., birthday, anniversary). Furthermore, the analysis unit can refer to past voice data and analyze it to grasp a long-term trend. Thereby, by determining the priority based on the collection time, the latest situation can be quickly grasped. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the analysis unit can input the collection time of the voice data to the generation AI and cause the generation AI to execute the determination of the priority.

[0088] The analysis unit can adjust the order of analysis based on the relevance of the voice data during analysis. The analysis unit, for example, evaluates the relevance of the voice data. For example, the analysis unit can evaluate the relevance based on the content of the voice data and the state of the speaker. Next, the analysis unit adjusts the order of analysis based on the relevance. For example, the analysis unit prioritizes analysis of important voice data (e.g., crying, coughing) to quickly grasp the situation. In addition, the analysis unit prioritizes analysis of highly relevant voice data (e.g., talking, laughter) to grasp the overall situation. Furthermore, the analysis unit can analyze less relevant voice data (e.g., background sound) later. As a result, important data can be analyzed preferentially by adjusting the order based on the relevance. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input the relevance of the voice data to the generation AI and cause the generation AI to adjust the order of analysis.

[0089] The notification unit can estimate the user's emotion and adjust the notification expression method based on the estimated user's emotion. The notification unit, for example, estimates the user's emotion when making a notification. For example, the notification unit can analyze voice data and estimate whether the user is relaxed or stressed. Next, the notification unit adjusts the notification expression method based on the estimated user's emotion. For example, the notification unit provides detailed notification content and notifies in an easy-to-understand format when the user is relaxed. Also, the notification unit can provide concise notification content and notify in a format that can be quickly understood when the user is stressed. Furthermore, the notification unit can provide short notification content including only the main points when the user is in a hurry. Thereby, by adjusting the expression method based on the emotion, the notification content can be provided in a more easily understandable manner. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the notification unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the notification unit can input voice data to the generation AI and have the generation AI estimate the user's emotions and adjust the way the notification is expressed.

[0090] The notification unit can adjust the level of detail of the notification based on the importance of the analysis result at the time of notification. The notification unit, for example, evaluates the importance of the analysis result. For example, the notification unit can evaluate the importance based on the content and urgency of the analysis result. Next, the notification unit adjusts the level of detail of the notification based on the importance. For example, the notification unit notifies in detail an analysis result with high importance (e.g., signs of illness) and proposes specific measures. In addition, the notification unit can briefly notify an analysis result with low importance (e.g., daily situation) to grasp the overall trend. Furthermore, the notification unit can appropriately notify an analysis result with medium importance (e.g., looking lonely) to grasp the situation. This enables efficient notification by adjusting the level of detail based on the importance. Part or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the analysis result to the generation AI and cause the generation AI to evaluate the importance and adjust the level of detail.

[0091] The notification unit can apply different notification algorithms depending on the category of the analysis result at the time of notification. The notification unit, for example, classifies the categories of the analysis result. For example, the notification unit can classify the analysis result into categories such as health condition, emergency call, and emotional state. Next, the notification unit applies different notification algorithms depending on the category. For example, the notification unit applies an emergency notification algorithm to the analysis result regarding the signs of illness and notifies promptly. In addition, the notification unit can apply a periodic notification algorithm to the analysis result regarding daily situations and notify periodically. Furthermore, the notification unit can apply an emotion notification algorithm to the analysis result regarding the emotional state and notify at an appropriate time. Thereby, by applying a notification algorithm according to the category, it is possible to notify at an appropriate time. Part or all of the above-mentioned processing in the notification unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the notification unit can input the analysis result to the generation AI and cause the generation AI to execute the classification of the category and the application of the notification algorithm.

[0092] The notification unit can estimate the user's emotion and adjust the length of the notification based on the estimated user's emotion. The notification unit, for example, estimates the user's emotion when making a notification. For example, the notification unit can analyze voice data and estimate whether the user is relaxed or stressed. Next, the notification unit adjusts the length of the notification based on the estimated user's emotion. For example, the notification unit can provide detailed notification content and perform a longer notification when the user is relaxed. Also, the notification unit can provide concise notification content and perform a shorter notification when the user is stressed. Furthermore, the notification unit can provide short notification content including only the main points when the user is in a hurry. Thereby, by adjusting the length of the notification based on the emotion, it is possible to provide notification content suitable for the user. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the notification unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the notification unit can input voice data to the generation AI and have the generation AI estimate the user's emotions and adjust the length of the notification.

[0093] The notification unit can determine the priority of the notification based on the collection time of the analysis result at the time of notification. The notification unit, for example, evaluates the collection time of the analysis result. For example, the notification unit can determine the priority based on the collection time of the analysis result. Next, the notification unit determines the priority of the notification based on the collection time. For example, the notification unit prioritizes notification of the analysis result collected recently to grasp the latest situation. In addition, the notification unit can prioritize notification of the analysis result related to a specific event (e.g., birthday, anniversary). Furthermore, the notification unit can refer to past analysis results and notify to grasp long-term trends. Thereby, by determining the priority based on the collection time, the latest situation can be quickly grasped. Part or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the collection time of the analysis result to the generation AI and cause the generation AI to execute the determination of the priority.

[0094] The notification unit can adjust the order of notifications based on the relevance of the analysis results at the time of notification. The notification unit, for example, evaluates the relevance of the analysis results. For example, the notification unit can evaluate the relevance based on the contents of the analysis results and the state of the speaker. Next, the notification unit adjusts the order of notifications based on the relevance. For example, the notification unit prioritizes notification of important analysis results (e.g., signs of illness) to enable a quick response. In addition, the notification unit prioritizes notification of highly relevant analysis results (e.g., looking lonely) to enable the situation to be grasped. Furthermore, the notification unit can postpone notification of less relevant analysis results (e.g., daily situations). As a result, important data can be prioritized by adjusting the order based on the relevance. Part or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the relevance of the analysis results to the generation AI and cause the generation AI to adjust the order of notifications.

[0095] The collection unit can estimate the user's emotion and adjust the type of sensor data to be collected based on the estimated user's emotion. The collection unit, for example, estimates the user's emotion when collecting sensor data. For example, the collection unit can analyze voice data and estimate whether the user is relaxed or stressed. Next, the collection unit adjusts the type of sensor data to be collected based on the estimated user's emotion. For example, when the user is relaxed, the collection unit preferentially collects laughter and talking. Also, when the user is stressed, the collection unit can preferentially collect voice data such as crying and coughing. Furthermore, when the user is in a hurry, the collection unit can preferentially collect voice data that can be collected in a short time. This allows more appropriate data to be collected by adjusting the type of sensor data based on emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the collection unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the collection unit can input voice data to the generation AI and cause the generation AI to estimate the user's emotions and adjust the type of sensor data.

[0096] The collection unit can select the optimal collection method by referring to the past data of the family when collecting the sensor data. The collection unit, for example, refers to the past data of the family when collecting the sensor data. For example, the collection unit confirms from the past sensor data that a lot of data occurs in a specific time period, and concentrates collection on that time period. The collection unit can also analyze the past sensor data and preferentially collect data related to a specific event (e.g., during exercise, during rest). Furthermore, the collection unit can identify a specific emotional state (e.g., a time period when the person is relaxed) from the past sensor data, and concentrate collection on that time period. This makes it possible to select the optimal collection method by referring to the past data. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit can input past data to the generation AI and cause the generation AI to select the optimal collection method.

[0097] The collection unit can estimate the user's emotion and determine the priority of the sensor data to be collected based on the estimated user's emotion. The collection unit, for example, estimates the user's emotion when collecting sensor data. For example, the collection unit can analyze voice data and estimate whether the user is relaxed or stressed. Next, the collection unit determines the priority of the sensor data to be collected based on the estimated user's emotion. For example, the collection unit preferentially collects data from a temperature sensor or a motion sensor when the user is relaxed. Also, the collection unit can preferentially collect data from a heart rate sensor or a breathing sensor when the user is stressed. Furthermore, the collection unit can preferentially collect data from a motion sensor and quickly analyze it when the user is in a hurry. This allows important data to be preferentially collected by determining the priority based on emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit may input voice data to the generation AI and cause the generation AI to estimate the user's emotions and determine the priority order of the sensor data.

[0098] The collection unit can preferentially collect highly relevant data in consideration of the geographical location information of the family when collecting the sensor data. The collection unit, for example, considers the geographical location information of the family when collecting the sensor data. For example, when the family is in a specific place (e.g., living room), the collection unit preferentially collects sensor data generated in that place. Furthermore, when the family is in a specific place (e.g., kitchen), the collection unit can preferentially collect sensor data generated in that place. Furthermore, when the family is in a specific place (e.g., bedroom), the collection unit can preferentially collect sensor data generated in that place. In this way, by considering the geographical location information, highly relevant data can be preferentially collected. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit can input the geographical location information to the generation AI and cause the generation AI to collect highly relevant data.

[0099] The analysis unit can estimate the user's emotion and adjust the expression method of the analysis based on the estimated user's emotion. The analysis unit estimates the user's emotion, for example, when analyzing voice data. For example, the analysis unit can analyze the voice data and estimate whether the user is relaxed or stressed. Next, the analysis unit adjusts the expression method of the analysis based on the estimated user's emotion. For example, when the user is relaxed, the analysis unit displays the analysis result in detail and provides it in an easy-to-understand format. In addition, when the user is stressed, the analysis unit can display the analysis result concisely and provide it in a format that can be quickly understood. Furthermore, when the user is in a hurry, the analysis unit can display only the main points of the analysis result and provide it in a format that can be quickly understood. Thereby, by adjusting the expression method based on the emotion, the analysis result can be provided in a more easily understandable manner. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the analysis unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the analysis unit may input voice data to the generation AI and cause the generation AI to estimate the user's emotions and adjust the expression method of the analysis.

[0100] The analysis unit can adjust the level of detail of the analysis based on the importance of the voice data during analysis. The analysis unit, for example, evaluates the importance of the voice data. For example, the analysis unit can evaluate the importance based on the contents of the voice data and the state of the speaker. Next, the analysis unit adjusts the level of detail of the analysis based on the importance. For example, the analysis unit analyzes voice data with high importance (e.g., crying, coughing) in detail to grasp the specific situation. In addition, the analysis unit can briefly analyze voice data with low importance (e.g., daily conversation) to grasp the overall trend. Furthermore, the analysis unit can moderately analyze voice data with medium importance (e.g., laughter) to grasp the situation. This allows efficient analysis by adjusting the level of detail based on the importance. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input voice data to the generation AI and cause the generation AI to evaluate the importance and adjust the level of detail.

[0101] The analysis unit can estimate the user's emotion and adjust the length of the analysis based on the estimated user's emotion. The analysis unit, for example, estimates the user's emotion when analyzing the voice data. For example, the analysis unit can analyze the voice data and estimate whether the user is relaxed or stressed. Next, the analysis unit adjusts the length of the analysis based on the estimated user's emotion. For example, the analysis unit can provide a detailed analysis result and generate a longer report when the user is relaxed. Also, the analysis unit can provide a concise analysis result and generate a shorter report when the user is stressed. Furthermore, the analysis unit can provide a short analysis result including only the main points when the user is in a hurry. Thereby, by adjusting the length of the analysis based on the emotion, an analysis result suitable for the user can be provided. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input voice data to the generation AI and have the generation AI estimate the user's emotions and adjust the length of the analysis.

[0102] During analysis, the analysis unit can determine the priority of analysis based on the collection time of the voice data. The analysis unit, for example, evaluates the collection time of the voice data. For example, the analysis unit can determine the priority based on the collection time of the voice data. Next, the analysis unit determines the priority of analysis based on the collection time. For example, the analysis unit preferentially analyzes recently collected voice data to grasp the latest situation. In addition, the analysis unit can preferentially analyze voice data related to a specific event (e.g., birthday, anniversary). Furthermore, the analysis unit can refer to past voice data and analyze it to grasp a long-term trend. Thereby, by determining the priority based on the collection time, the latest situation can be quickly grasped. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the analysis unit can input the collection time of the voice data to the generation AI and cause the generation AI to execute the determination of the priority.

[0103] The notification unit can estimate the user's emotion and adjust the notification expression method based on the estimated user's emotion. The notification unit, for example, estimates the user's emotion when making a notification. For example, the notification unit can analyze voice data and estimate whether the user is relaxed or stressed. Next, the notification unit adjusts the notification expression method based on the estimated user's emotion. For example, the notification unit provides detailed notification content and notifies in an easy-to-understand format when the user is relaxed. Also, the notification unit can provide concise notification content and notify in a format that can be quickly understood when the user is stressed. Furthermore, the notification unit can provide short notification content including only the main points when the user is in a hurry. Thereby, by adjusting the expression method based on the emotion, the notification content can be provided in a more easily understandable manner. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the notification unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the notification unit can input voice data to the generation AI and have the generation AI estimate the user's emotions and adjust the way the notification is expressed.

[0104] The notification unit can adjust the level of detail of the notification based on the importance of the analysis result at the time of notification. The notification unit, for example, evaluates the importance of the analysis result. For example, the notification unit can evaluate the importance based on the content and urgency of the analysis result. Next, the notification unit adjusts the level of detail of the notification based on the importance. For example, the notification unit notifies in detail an analysis result with high importance (e.g., signs of illness) and proposes specific measures. In addition, the notification unit can briefly notify an analysis result with low importance (e.g., daily situation) to grasp the overall trend. Furthermore, the notification unit can appropriately notify an analysis result with medium importance (e.g., looking lonely) to grasp the situation. This enables efficient notification by adjusting the level of detail based on the importance. Part or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the analysis result to the generation AI and cause the generation AI to evaluate the importance and adjust the level of detail.

[0105] The notification unit can estimate the user's emotion and adjust the length of the notification based on the estimated user's emotion. The notification unit, for example, estimates the user's emotion when making a notification. For example, the notification unit can analyze voice data and estimate whether the user is relaxed or stressed. Next, the notification unit adjusts the length of the notification based on the estimated user's emotion. For example, the notification unit can provide detailed notification content and perform a longer notification when the user is relaxed. Also, the notification unit can provide concise notification content and perform a shorter notification when the user is stressed. Furthermore, the notification unit can provide short notification content including only the main points when the user is in a hurry. Thereby, by adjusting the length of the notification based on the emotion, it is possible to provide notification content suitable for the user. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the notification unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the notification unit can input voice data to the generation AI and have the generation AI estimate the user's emotions and adjust the length of the notification.

[0106] The notification unit can determine the priority of the notification based on the collection time of the analysis result at the time of notification. The notification unit, for example, evaluates the collection time of the analysis result. For example, the notification unit can determine the priority based on the collection time of the analysis result. Next, the notification unit determines the priority of the notification based on the collection time. For example, the notification unit prioritizes notification of the analysis result collected recently to grasp the latest situation. In addition, the notification unit can prioritize notification of the analysis result related to a specific event (e.g., birthday, anniversary). Furthermore, the notification unit can refer to past analysis results and notify to grasp long-term trends. Thereby, by determining the priority based on the collection time, the latest situation can be quickly grasped. Part or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the collection time of the analysis result to the generation AI and cause the generation AI to execute the determination of the priority.

[0107] The notification unit can adjust the order of notifications based on the relevance of the analysis results at the time of notification. The notification unit, for example, evaluates the relevance of the analysis results. For example, the notification unit can evaluate the relevance based on the contents of the analysis results and the state of the speaker. Next, the notification unit adjusts the order of notifications based on the relevance. For example, the notification unit prioritizes notification of important analysis results (e.g., signs of illness) to enable a quick response. In addition, the notification unit prioritizes notification of highly relevant analysis results (e.g., looking lonely) to enable the situation to be grasped. Furthermore, the notification unit can postpone notification of less relevant analysis results (e.g., daily situations). As a result, important data can be prioritized by adjusting the order based on the relevance. Part or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the relevance of the analysis results to the generation AI and cause the generation AI to adjust the order of notifications.

[0108] The collection unit can estimate the user's emotion and adjust the type of sensor data to be collected based on the estimated user's emotion. The collection unit, for example, estimates the user's emotion when collecting sensor data. For example, the collection unit can analyze voice data and estimate whether the user is relaxed or stressed. Next, the collection unit adjusts the type of sensor data to be collected based on the estimated user's emotion. For example, the collection unit preferentially collects data from a temperature sensor or a motion sensor when the user is relaxed. Also, the collection unit can preferentially collect data from a heart rate sensor or a breathing sensor when the user is stressed. Furthermore, the collection unit can preferentially collect data from a motion sensor and quickly analyze it when the user is in a hurry. This allows more appropriate data to be collected by adjusting the type of sensor data based on the emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit may input voice data to the generation AI and cause the generation AI to estimate the user's emotions and adjust the type of sensor data.

[0109] The collection unit can select the optimal collection method by referring to the past data of the family when collecting the sensor data. The collection unit, for example, refers to the past data of the family when collecting the sensor data. For example, the collection unit confirms from the past sensor data that a lot of data occurs in a specific time period, and concentrates collection on that time period. The collection unit can also analyze the past sensor data and preferentially collect data related to a specific event (e.g., during exercise, during rest). Furthermore, the collection unit can identify a specific emotional state (e.g., a time period when the person is relaxed) from the past sensor data, and concentrate collection on that time period. This makes it possible to select the optimal collection method by referring to the past data. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit can input past data to the generation AI and cause the generation AI to select the optimal collection method.

[0110] The collection unit can estimate the user's emotion and determine the priority of the sensor data to be collected based on the estimated user's emotion. The collection unit, for example, estimates the user's emotion when collecting sensor data. For example, the collection unit can analyze voice data and estimate whether the user is relaxed or stressed. Next, the collection unit determines the priority of the sensor data to be collected based on the estimated user's emotion. For example, the collection unit preferentially collects data from a temperature sensor or a motion sensor when the user is relaxed. Also, the collection unit can preferentially collect data from a heart rate sensor or a breathing sensor when the user is stressed. Furthermore, the collection unit can preferentially collect data from a motion sensor and quickly analyze it when the user is in a hurry. This allows important data to be preferentially collected by determining the priority based on emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit may input voice data to the generation AI and cause the generation AI to estimate the user's emotions and determine the priority order of the sensor data.

[0111] The collection unit can preferentially collect highly relevant data in consideration of the geographical location information of the family when collecting the sensor data. The collection unit, for example, considers the geographical location information of the family when collecting the sensor data. For example, when the family is in a specific place (e.g., living room), the collection unit preferentially collects sensor data generated in that place. Furthermore, when the family is in a specific place (e.g., kitchen), the collection unit can preferentially collect sensor data generated in that place. Furthermore, when the family is in a specific place (e.g., bedroom), the collection unit can preferentially collect sensor data generated in that place. In this way, by considering the geographical location information, highly relevant data can be preferentially collected. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit can input the geographical location information to the generation AI and cause the generation AI to collect highly relevant data.

[0112] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0113] The collection unit can learn the life rhythms of family members and automatically adjust the optimal collection timing. For example, the collection unit can start collecting voice data when family members usually wake up and stop collecting when they go to bed. The collection unit can also focus collection on times when family members perform specific activities (e.g., eating, exercising). Furthermore, the collection unit can adjust the collection timing on special days such as weekends and holidays based on the family members' life rhythms. This allows more accurate data to be collected by optimizing the collection timing based on the life rhythms.

[0114] The analysis unit can visually display the analysis results of the voice data. For example, the analysis unit can display the emotional state of family members in graphs and charts to make it easier to understand visually. The analysis unit can also display the tone and frequency of the voice data in different colors to make it possible to grasp changes in emotions at a glance. Furthermore, the analysis unit can display the analysis results in a timeline format to visually track changes in emotions over time. This allows the analysis results to be more intuitively understood through visual displays.

[0115] The notification unit can suggest specific action plans for family members based on the analysis results. For example, if a family member looks lonely, the notification unit can suggest specific actions such as "calling" or "visiting" relatives. In addition, if a family member shows signs of illness, the notification unit can send a notification encouraging the family member to visit a medical institution. Furthermore, if a family member looks relaxed, the notification unit can suggest activities to maintain relaxation (e.g., walking, reading). This makes it possible to respond quickly and appropriately by suggesting specific action plans.

[0116] The collection unit can collect health data (e.g., heart rate, blood pressure) of family members and use it to understand their living conditions. For example, the collection unit can collect heart rate data of family members using a heart rate sensor and evaluate their stress levels. The collection unit can also collect blood pressure data of family members using a blood pressure sensor and monitor their health conditions. Furthermore, the collection unit can collect sleep data of family members using a sleep sensor and evaluate their sleep quality. This makes it possible to understand the living conditions more comprehensively by collecting health data.

[0117] The analysis unit can analyze the family's living conditions in combination with other data sources (e.g., weather information, calendar information). For example, the analysis unit can refer to weather information and analyze whether the family spends more time indoors on bad weather days. The analysis unit can also refer to calendar information and analyze changes in emotions related to specific events (e.g., birthdays, anniversaries). Furthermore, the analysis unit can refer to traffic information and analyze the impact of fluctuations in commuting time on the family's stress level. This allows for more accurate analysis by combining with other data sources.

[0118] The collection unit can estimate the user's emotion and adjust the type of voice data to be collected based on the estimated user's emotion. For example, when the user is relaxed, the collection unit can preferentially collect laughter and talking. When the user is stressed, the collection unit can preferentially collect voice data such as crying and coughing. Furthermore, when the user is in a hurry, the collection unit can preferentially collect voice data that can be collected in a short time. In this way, by adjusting the type of voice data to be collected based on emotion, more appropriate data can be collected.

[0119] The analysis unit can estimate the user's emotions and adjust the way the analysis is presented based on the estimated user's emotions. For example, when the user is relaxed, the analysis unit can display the analysis results in detail and provide them in an easy-to-understand format. When the user is stressed, the analysis unit can display the analysis results concisely and provide them in a format that can be quickly understood. When the user is in a hurry, the analysis unit can display only the main points of the analysis results and provide them in a format that can be quickly understood. In this way, by adjusting the way the analysis is presented based on emotions, the analysis results can be presented in a more easy-to-understand format.

[0120] The notification unit can estimate the user's emotion and adjust the notification expression method based on the estimated user's emotion. For example, when the user is relaxed, the notification unit can provide detailed notification content and notify the user in an easy-to-understand format. When the user is stressed, the notification unit can provide concise notification content and notify the user in a format that can be quickly understood. Furthermore, when the user is in a hurry, the notification unit can provide short notification content including only the main points. In this way, by adjusting the expression method based on the emotion, the notification content can be provided in a more easy-to-understand manner.

[0121] The collection unit can estimate the user's emotion and determine the priority of the sensor data to be collected based on the estimated emotion of the user. For example, when the user is relaxed, the collection unit can preferentially collect data from the temperature sensor and the motion sensor. When the user is stressed, the collection unit can preferentially collect data from the heart rate sensor and the breathing sensor. When the user is in a hurry, the collection unit can preferentially collect data from the motion sensor and quickly analyze it. Thus, by determining the priority based on the emotion, important data can be preferentially collected.

[0122] The analysis unit can estimate the user's emotions and adjust the length of the analysis based on the estimated user's emotions. For example, the analysis unit can provide detailed analysis results and generate a longer report when the user is relaxed. Also, the analysis unit can provide concise analysis results and generate a shorter report when the user is stressed. Furthermore, the analysis unit can provide a short analysis result including only the main points when the user is in a hurry. In this way, by adjusting the length of the analysis based on emotions, it is possible to provide an analysis result suitable for the user.

[0123] The process flow of the second embodiment will be briefly described below.

[0124] Step 1: The collection unit collects voice data. The voice data includes talking, laughing, crying, coughing, etc. The collection unit collects the voice data in real time using voice sensors installed in the house. The collection unit can also collect data from other sensors such as motion sensors and temperature sensors. Step 2: The analysis unit uses the generation AI to analyze the voice data collected by the collection unit. The analysis is carried out using voice recognition technology and emotion analysis algorithms. For example, the tone and frequency of speech, the presence or absence of laughter, the frequency of crying, etc. are analyzed to determine whether the person is living a healthy life, seems lonely, or is at risk of illness, depression, or mental disorder. Step 3: The notification unit issues a notification based on the analysis results obtained by the analysis unit. The notification is given by email notification, app notification, voice notification, or other methods. For example, based on the analysis results, a notification such as "You look lonely" can be sent to relatives. The notification unit can also generate specific notification content based on the analysis results and send it to relatives.

[0125] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0126] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0127] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0128] Each of the multiple elements including the above-mentioned collection unit, analysis unit, and notification unit is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the collection unit collects voice data in the house in real time using a voice sensor of the smart device 14. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the voice data using a generation AI to determine the living situation of the family. The notification unit is realized, for example, by the control unit 46A of the smart device 14, and transmits a notification to relatives based on the analysis result. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0129] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0130] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0131] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0132] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0133] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0134] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0135] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0136] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0137] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0138] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0139] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0140] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0141] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0142] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0143] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0144] Each of the multiple elements including the above-mentioned collection unit, analysis unit, and notification unit is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the collection unit collects voice data in the house in real time using a voice sensor of the smart glasses 214. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the voice data using a generation AI to determine the living situation of the family. The notification unit is realized, for example, by the control unit 46A of the smart glasses 214, and transmits a notification to relatives based on the analysis result. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0145] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0146] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0147] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0148] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0149] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0150] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0151] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0152] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0153] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0154] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0155] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0156] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0157] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0158] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0159] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0160] Each of the multiple elements including the above-mentioned collection unit, analysis unit, and notification unit is realized, for example, by at least one of the headset type terminal 314 and the data processing device 12. For example, the collection unit uses a voice sensor of the headset type terminal 314 to collect voice data in the house in real time. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the voice data using a generation AI to determine the living conditions of the family. The notification unit is realized, for example, by the control unit 46A of the headset type terminal 314, and transmits a notification to relatives based on the analysis result. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0161] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0162] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0163] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0164] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0165] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0166] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0167] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0168] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0169] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0170] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0171] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0172] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0173] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0174] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0175] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0176] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0177] Each of the multiple elements including the above-mentioned collection unit, analysis unit, and notification unit is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the collection unit uses a voice sensor of the robot 414 to collect voice data in the house in real time. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the voice data using a generation AI to determine the living conditions of the family. The notification unit is realized, for example, by the control unit 46A of the robot 414, and transmits a notification to relatives based on the analysis result. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0178] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0179] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0180] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0181] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out on emotion map 400 you go, the more visible the emotions become (the more they are expressed in behavior).

[0182] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0183] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0184] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0185] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0186] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0187] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0188] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0189] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0190] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0191] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0192] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0193] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0194] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0195] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0196] (Appendix 1) A collection unit that collects voice data from within a house where a family lives; An analysis unit that analyzes the voice data collected by the collection unit using a generation AI; A notification unit that issues a notification based on the analysis result obtained by the analysis unit. A system characterized by: (Appendix 2) The collecting unit includes: Collect talking, laughing, crying, coughing and other audio data 2. The system of claim 1, (Appendix 3) The analysis unit is Voice data is analyzed to determine whether a person is living a healthy life, feeling lonely, or at risk of illness, depression, or mental disorders. 2. The system of claim 1, (Appendix 4) The notification unit is Notify relevant parties based on the analysis results 2. The system of claim 1, (Appendix 5) The notification unit is Generate notification content specifically 2. The system of claim 1, (Appendix 6) The collecting unit includes: It also collects data from motion sensors, temperature sensors, etc. 2. The system of claim 1, (Appendix 7) The collecting unit includes: To estimate a user's emotion and adjust a timing of collecting voice data based on the estimated user's emotion. 2. The system of claim 1, (Appendix 8) The collecting unit includes: When collecting voice data, select the appropriate collection method by referring to past voice data of family members. 2. The system of claim 1, (Appendix 9) The collecting unit includes: When collecting voice data, filtering is performed based on family behavior patterns. 2. The system of claim 1, (Appendix 10) The collecting unit includes: Estimate user emotions and prioritize voice data to be collected based on the estimated user emotions. 2. The system of claim 1, (Appendix 11) The collecting unit includes: When collecting voice data, prioritize collection of relevant data based on the family’s geographic location 2. The system of claim 1, (Appendix 12) The collecting unit includes: When collecting voice data, analyze family members' social media usage and collect related voice data. 2. The system of claim 1, (Appendix 13) The analysis unit is Infer user emotions and adjust the presentation of analytics based on the inferred user emotions 2. The system of claim 1, (Appendix 14) The analysis unit is During analysis, adjust the level of analysis detail based on the importance of the audio data 2. The system of claim 1, (Appendix 15) The analysis unit is During analysis, different analysis algorithms are applied depending on the category of audio data. 2. The system of claim 1, (Appendix 16) The analysis unit is Infer user sentiment and adjust the length of the analysis based on the inferred user sentiment 2. The system of claim 1, (Appendix 17) The analysis unit is During analysis, prioritize analysis based on when the audio data was collected 2. The system of claim 1, (Appendix 18) The analysis unit is During analysis, adjust the order of analysis based on the relevance of the audio data 2. The system of claim 1, (Appendix 19) The notification unit is Infer user emotions and adjust notification presentation based on the inferred user emotions 2. The system of claim 1, (Appendix 20) The notification unit is When notifying, adjust the level of detail of notifications based on the importance of the analysis results 2. The system of claim 1, (Appendix 21) The notification unit is During notification, apply different notification algorithms depending on the analysis result category 2. The system of claim 1, (Appendix 22) The notification unit is Infer user emotion and adjust notification length based on the estimated user emotion 2. The system of claim 1, (Appendix 23) The notification unit is When notifications are sent, they are prioritized based on when the analysis results were collected 2. The system of claim 1, (Appendix 24) The notification unit is At the time of notification, adjust the order of notifications based on the relevance of the analysis results 2. The system of claim 1, (Appendix 25) The collecting unit includes: Inferring a user's emotion and adjusting the type of sensor data to be collected based on the estimated user's emotion 2. The system of claim 1, (Appendix 26) The collecting unit includes: When collecting sensor data, the optimal collection method is selected by referring to the family's past data. 2. The system of claim 1, (Appendix 27) The collecting unit includes: Estimate user emotions and prioritize sensor data to be collected based on the estimated user emotions. 2. The system of claim 1, (Appendix 28) The collecting unit includes: When collecting sensor data, the system takes into account the geographic location information of family members to prioritize the collection of relevant data. 2. The system of claim 1, (Appendix 29) The analysis unit is Infer user emotions and adjust the presentation of analytics based on the inferred user emotions 2. The system of claim 1, (Appendix 30) The analysis unit is During analysis, adjust the level of analysis detail based on the importance of the audio data 2. The system of claim 1, (Appendix 31) The analysis unit is Infer user sentiment and adjust the length of the analysis based on the inferred user sentiment 2. The system of claim 1, (Appendix 32) The analysis unit is During analysis, prioritize analysis based on when the audio data was collected 2. The system of claim 1, (Appendix 33) The notification unit is Infer user emotions and adjust notification presentation based on the inferred user emotions 2. The system of claim 1, (Appendix 34) The notification unit is When notifying, adjust the level of detail of notifications based on the importance of the analysis results 2. The system of claim 1, (Appendix 35) The notification unit is Infer user emotion and adjust notification length based on the estimated user emotion 2. The system of claim 1, (Appendix 36) The notification unit is When notifications are sent, they are prioritized based on when the analysis results were collected 2. The system of claim 1, (Appendix 37) The notification unit is At the time of notification, adjust the order of notifications based on the relevance of the analysis results 2. The system of claim 1, (Appendix 38) The collecting unit includes: Inferring a user's emotion and adjusting the type of sensor data to be collected based on the estimated user's emotion 2. The system of claim 1, (Appendix 39) The collecting unit includes: When collecting sensor data, the optimal collection method is selected by referring to the family's past data. 2. The system of claim 1, (Appendix 40) The collecting unit includes: Estimate user emotions and prioritize sensor data to be collected based on the estimated user emotions. 2. The system of claim 1, (Appendix 41) The collecting unit includes: When collecting sensor data, the system takes into account the geographic location information of family members to prioritize the collection of relevant data. 2. The system of claim 1, [Explanation of symbols]

[0197] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A collection unit that collects voice data from within a house where a family lives; An analysis unit that analyzes the voice data collected by the collection unit using a generation AI; A notification unit that notifies relevant persons based on the analysis result obtained by the analysis unit. A system characterized by:

2. The collecting unit includes: Collects audio data of talking, laughing, crying, and coughing 2. The system of claim 1 .

3. The analysis unit is Voice data is analyzed to determine whether a person is living a healthy life, feeling lonely, or at risk of illness, depression, or mental disorders.

2. The system of claim 1 .

4. The notification unit is Adjusting the order of notifications based on the relevance of the analysis results of the analysis unit 2. The system of claim 1 .

5. The notification unit is Infer user emotion and adjust notification length based on the estimated user emotion 2. The system of claim 1 .

6. The collecting unit includes: Prioritizing collection of relevant data based on the geographic location information of the family members 2. The system of claim 1 .

7. The collecting unit includes: To estimate a user's emotion and adjust a timing of collecting voice data based on the estimated user's emotion.

2. The system of claim 1 .

8. The collecting unit includes: Select an appropriate collection method by referring to the past voice data of the family member.

2. The system of claim 1 .

Images