System

Abstract

To provide a system according to the embodiment which generally assists a decrease in vision, hearing, and equilibrium sense caused by aging of the sensory system of the elderly.SOLUTION: A system according to the embodiment includes a vision adjustment part, a hearing sensation assistance part, a visual field sharing part, a life management part, and a care assistance part. The vision adjustment part adjusts eyesight in real time. The hearing sensation assistance part amplifies sound in the surroundings. The visual field sharing part communicates with a supporter in a remote place in real time and shares the user's visual field. The life management part supports the user's daily life. The care assistance part assists a decrease in equilibrium sense or athletic ability.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology does not adequately provide a means of comprehensively assisting the elderly with the decline in vision, hearing, and balance that accompanies the aging of the sensory systems, and there is room for improvement.

[0005] The system according to the embodiment aims to provide comprehensive assistance to the elderly for the decline in vision, hearing, and balance that accompanies the aging of the sensory systems. [Means for solving the problem]

[0006] The system according to the embodiment includes a vision adjustment unit, an auditory assistance unit, a visual field sharing unit, a life management unit, and a care assistance unit. The vision adjustment unit adjusts the visual field in real time. The auditory assistance unit amplifies surrounding sounds. The visual field sharing unit communicates with a remote supporter in real time and shares the user's visual field. The life management unit supports the user's daily life. The care assistance unit compensates for a decline in balance or motor skills. Effect of the Invention

[0007] The system according to the embodiment can comprehensively assist the elderly in suffering from the decline in vision, hearing, and balance that accompanies the aging of the sensory system. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The system according to the embodiment of the present invention is an AR smart glass system for assisting the decline in "vision" and "hearing" associated with the aging of the sensory organs of the elderly. This AR smart glass system is equipped with the functions of vision adjustment, hearing assistance, remote assistance, life management, and care assistance, and can compensate for the decline in physical functions associated with the aging of the sensory organs of the elderly. For example, as a vision adjustment function, the AR smart glass adjusts the user's vision in real time. In addition to correcting vision, visual information can be enlarged and displayed using AR technology. Next, as a hearing assistance function, the AR smart glass amplifies surrounding sounds to assist the user's hearing. This allows the user to hear sounds clearly. Furthermore, as a remote assistance function, the AR smart glass communicates with a remote supporter in real time, and the supporter can provide assistance while sharing the user's field of view. For example, a family member or caregiver in a remote location can give appropriate instructions while looking at the user's field of view. In addition, as a life management function, the AR smart glass supports the user's daily life. For example, it can notify the time to take medicine and display a schedule. Finally, as a care assistance function, the AR smart glass assists the user's decline in physical function. For example, AR technology can be used to display visual guides for balancing to compensate for the decline in balance and motor skills, allowing the AR smart glasses system to compensate for the decline in physical functions associated with the aging of the sensory system in the elderly.

[0029] The AR smart glasses system according to the embodiment includes a vision adjustment unit, an auditory assistance unit, a visual field sharing unit, a life management unit, and a care assistance unit. The vision adjustment unit adjusts the user's visual acuity in real time. The vision adjustment unit includes, for example, a focus adjustment function to correct the visual acuity. The vision adjustment unit can also enlarge and display visual information using AR technology. For example, the vision adjustment unit automatically adjusts the focus according to the user's visual acuity to display the visual information clearly. Furthermore, the vision adjustment unit includes a brightness adjustment function and can automatically adjust the brightness of the visual information according to the surrounding light environment. The auditory assistance unit amplifies surrounding sounds and assists the user's hearing. For example, the auditory assistance unit amplifies surrounding sounds in decibel units to enable the user to hear sounds clearly. Furthermore, the auditory assistance unit includes a noise canceling function and can reduce surrounding noise. For example, the auditory assistance unit analyzes surrounding sounds in real time and emphasizes and amplifies important sounds. The visual field sharing unit communicates with a remote supporter in real time and shares the user's visual field. The view sharing unit, for example, photographs the user's view using a high-resolution camera and transmits it to a supporter in a remote location. The view sharing unit can also share view information in real time using a communication protocol. For example, the view sharing unit transmits view information via the Internet, and the supporter can give appropriate instructions while viewing the user's view. The life management unit supports the user's daily life. The life management unit, for example, has a function of notifying the user of the time to take medicine. The life management unit can also manage the user's schedule and display the plan. For example, the life management unit visually displays the user's plan using a calendar display or a list display. The care support unit displays a visual guide for maintaining balance using AR technology to compensate for a decrease in balance and motor skills. The care support unit, for example, analyzes the user's movements in real time and displays a visual guide for maintaining balance. The care support unit also has an exercise support function and can support the user's motor skills. For example, the care support unit monitors the user's movements and suggests an appropriate exercise method. As a result, the AR smart glasses system according to the embodiment can compensate for the decline in physical functions that accompanies the aging of the sensory organs in the elderly.

[0030] The vision adjustment unit adjusts the user's vision in real time. The vision adjustment unit has, for example, a focus adjustment function to correct the vision. Specifically, the vision adjustment unit employs active lens technology to automatically adjust the focus according to the user's vision. This active lens can instantly change the focal length using liquid crystal or electrically controlled materials. The vision adjustment unit can also enlarge and display visual information using AR technology. For example, when a user reads small characters, the vision adjustment unit automatically enlarges the characters to make them easier to read. Furthermore, the vision adjustment unit has a brightness adjustment function and can automatically adjust the brightness of visual information according to the surrounding light environment. This allows the user to always check information in an optimal visual environment, regardless of day or night. The vision adjustment unit continuously monitors the user's vision data and makes adjustments as necessary, thereby reducing eye fatigue even with long-term use. Furthermore, the vision adjustment unit can periodically perform a vision test according to changes in the user's vision and provide optimal correction based on the results. This allows the vision adjustment unit to always keep the user's vision in an optimal state and provide a comfortable visual experience.

[0031] The hearing assistance unit amplifies surrounding sounds and assists the user's hearing. For example, the hearing assistance unit amplifies surrounding sounds in decibel units, allowing the user to hear the sounds clearly. Specifically, the hearing assistance unit collects surrounding sounds using a highly sensitive microphone and amplifies the sounds using digital signal processing technology. The hearing assistance unit also has a noise canceling function and can reduce surrounding noise. For example, the hearing assistance unit analyzes surrounding sounds in real time and emphasizes and amplifies important sounds. This allows the user to clearly hear necessary audio information even in a noisy environment. Furthermore, the hearing assistance unit can adjust the frequency characteristics of the sound according to the user's hearing characteristics. For example, for a user who has difficulty hearing high-pitched sounds, the high-pitched sounds are emphasized and amplified to optimize the overall sound balance. The hearing assistance unit continuously monitors the user's hearing data and makes adjustments as necessary, thereby reducing ear fatigue even with long-term use. Furthermore, the hearing assistance unit can periodically perform hearing tests according to changes in the user's hearing and provide optimal assistance based on the results. This allows the hearing assistance unit to always keep the user's hearing in an optimal state, providing a comfortable hearing experience.

[0032] The view sharing unit communicates with a remote supporter in real time and shares the user's view. The view sharing unit, for example, uses a high-resolution camera to capture the user's view and transmits it to the remote supporter. Specifically, the view sharing unit captures the user's view with a wide-angle lens, compresses the video, and transmits it via the Internet. The view sharing unit can also share view information in real time using a communication protocol. For example, the view sharing unit can use low-latency streaming technology to enable the supporter to check the user's view in almost real time. This allows the supporter to give appropriate instructions while viewing the user's view. Furthermore, the view sharing unit has a two-way communication function, and the supporter and the user can communicate by voice or text. This allows the supporter to provide specific instructions and advice to the user in real time. The view sharing unit encrypts and transmits the user's view information, so privacy protection is also ensured. This allows the view sharing unit to realize safe and efficient communication between the supporter and the user in a remote location and support the user's life.

[0033] The life management unit supports the user's daily life. The life management unit has, for example, a function of notifying the user of the time to take medicine. Specifically, the life management unit can manage the user's schedule and display the plan. For example, the life management unit visually displays the user's plan using a calendar display or a list display. Furthermore, the life management unit can monitor the user's health condition and provide health management advice as necessary. For example, the life management unit measures the user's heart rate and blood pressure, and issues an alert when an abnormality is detected. In addition, the life management unit can manage the user's diet and exercise records and make suggestions to support a balanced life. In this way, the life management unit can comprehensively support the user's daily life and realize a healthy and comfortable life. In addition, the life management unit can provide notifications and reminders customized based on the user's preferences and habits. For example, the life management unit provides support tailored to the user's lifestyle, such as a reminder to take medicine at a specific time and a suggestion for regular exercise. In this way, the life management unit can improve the user's quality of life and make daily life more convenient and comfortable.

[0034] The care assistant unit displays visual guides for maintaining balance using AR technology to compensate for the decline in balance and motor skills. Specifically, the care assistant unit analyzes the user's movements in real time and displays visual guides for maintaining balance. For example, if the user is about to lose balance while walking, the care assistant unit displays visual lines and arrows to provide instructions for the user to regain balance. The care assistant unit also has an exercise assistance function and can support the user's motor skills. For example, the care assistant unit monitors the user's movements and suggests appropriate exercise methods. This allows the user to safely exercise and maintain and improve their motor skills. Furthermore, the care assistant unit can accumulate the user's motion data and analyze long-term changes in motor skills. This allows the care assistant unit to detect the decline in the user's motor skills early and take appropriate measures. The care assistant unit provides important functions for supporting independence in daily life while ensuring the safety of the user. This allows the care assistant unit to improve the quality of life of elderly people and users with reduced motor skills, allowing them to live their daily lives with peace of mind.

[0035] The vision adjustment unit can enlarge and display visual information using AR technology. The vision adjustment unit, for example, uses AR technology to enlarge and display visual information. The AR technology can enlarge the user's visual information using an augmented reality display method. For example, the vision adjustment unit automatically enlarges visual information according to the user's visual acuity to reduce visual burden. In addition, the vision adjustment unit can optimize visual information according to the device used. For example, the vision adjustment unit uses a display of smart glasses to enlarge and display visual information. This makes it possible to compensate for a decrease in visual acuity by enlarging and displaying visual information.

[0036] The hearing assistance unit amplifies surrounding sounds and allows the user to hear the sounds clearly. For example, the hearing assistance unit amplifies surrounding sounds and allows the user to hear the sounds clearly. The hearing assistance unit amplifies sounds in decibel units to assist the user's hearing. The hearing assistance unit can also reduce surrounding noise using noise reduction technology. For example, the hearing assistance unit analyzes surrounding sounds in real time and emphasizes and amplifies important sounds. This makes it possible to compensate for a decrease in hearing by amplifying surrounding sounds.

[0037] The view sharing unit communicates with a remote supporter in real time, and the supporter can give instructions while viewing the user's view. The view sharing unit communicates with a remote supporter in real time, and the supporter can give instructions while viewing the user's view. The view sharing unit captures the user's view using a high-resolution camera and transmits it to the remote supporter. The view sharing unit can also share view information in real time using a communication protocol. For example, the view sharing unit transmits view information via the Internet, and the supporter can give appropriate instructions while viewing the user's view. This allows the remote supporter to provide appropriate support by sharing the user's view.

[0038] The life management unit can notify the user of the time to take medicine and display a schedule. The life management unit can, for example, notify the user of the time to take medicine and display a schedule. The life management unit notifies the user of the time to take medicine using an alarm sound or vibration. The life management unit can also visually display the user's schedule using a calendar display or a list display. For example, the life management unit manages the user's schedule and notifies the user at an appropriate time. In this way, it is possible to support daily life by notifying the user of the time to take medicine and displaying the schedule.

[0039] The care assistant unit can display a visual guide for maintaining balance using AR technology. The care assistant unit can display a visual guide for maintaining balance using AR technology, for example. The care assistant unit displays a visual guide for maintaining balance for the user using an arrow display or a line display. In addition, the care assistant unit can analyze the user's movements in real time and display appropriate guides. For example, the care assistant unit monitors the user's movements and displays a visual guide for maintaining balance. In this way, a decline in balance and motor skills can be compensated for by displaying a visual guide for maintaining balance using AR technology.

[0040] The vision adjustment unit can analyze the user's past vision data and select the optimal vision adjustment method. The vision adjustment unit can, for example, analyze the user's past vision data and select the optimal vision adjustment method. The vision adjustment unit selects the most effective vision correction method based on the user's vision test results and feedback. In addition, the vision adjustment unit can analyze the user's vision change pattern and adjust the vision at an appropriate time. For example, the vision adjustment unit monitors the user's vision data over the long term and updates the adjustment method according to changes in vision. In this way, the optimal vision adjustment method can be selected by analyzing past vision data.

[0041] The vision adjustment unit can adjust the display method of visual information based on the user's current activity content during vision adjustment. For example, the vision adjustment unit can adjust the display method of visual information based on the user's current activity content during vision adjustment. The vision adjustment unit analyzes the user's activity content in real time to provide optimal visual information. For example, when the user is reading, the vision adjustment unit enlarges and displays characters to make them easier to read. In addition, when the user is exercising, the vision adjustment unit can dynamically adjust the visual information according to the movement. Furthermore, when the user is cooking, the vision adjustment unit can visually highlight recipes and measurement information. This makes it possible to provide an optimal visual experience by customizing the display method of visual information based on the user's activity content.

[0042] The vision adjustment unit can provide appropriate visual information in consideration of the geographical location information of the user when adjusting the vision. The vision adjustment unit can provide appropriate visual information in consideration of the geographical location information of the user when adjusting the vision. The vision adjustment unit provides optimal visual information based on the geographical location information of the user. For example, when the user is outdoors, the vision adjustment unit adjusts brightness and contrast to optimize the visual information. In addition, when the user is indoors, the vision adjustment unit can adjust the visual information according to the lighting conditions. Furthermore, when the user is moving, the vision adjustment unit can dynamically update the visual information based on the geographical location. Thereby, optimal visual information can be provided by taking into account the geographical location information.

[0043] The vision adjustment unit can analyze the user's social media activity and provide related visual information when adjusting the vision. For example, the vision adjustment unit can analyze the user's social media activity and provide related visual information when adjusting the vision. The vision adjustment unit customizes the visual information based on the user's social media activity. For example, the vision adjustment unit customizes the visual information based on photos shared by the user on social media. In addition, the vision adjustment unit can personalize the visual information based on the content of the user's social media posts. Furthermore, the vision adjustment unit can analyze the user's social media activity history and provide related visual information. In this way, it is possible to provide related visual information by analyzing social media activity.

[0044] The hearing assistance unit can analyze the user's past hearing data and select the optimal sound amplification method. The hearing assistance unit can, for example, analyze the user's past hearing data and select the optimal sound amplification method. The hearing assistance unit selects the most effective sound amplification method based on the user's hearing test results and feedback. In addition, the hearing assistance unit can analyze the user's hearing fluctuation pattern and amplify the sound at an appropriate time. For example, the hearing assistance unit monitors the user's hearing data over the long term and updates the amplification method according to changes in hearing. In this way, the optimal sound amplification method can be selected by analyzing the past hearing data.

[0045] The hearing assistance unit can filter the sound based on the user's current environmental sound during hearing assistance. For example, the hearing assistance unit can filter the sound based on the user's current environmental sound during hearing assistance. The hearing assistance unit analyzes the surrounding noise level in real time and performs optimal sound filtering. For example, when the user is in a noisy environment, the hearing assistance unit activates a noise canceling function to block external noise. In addition, when the user is in a quiet environment, the hearing assistance unit can naturally amplify the surrounding sound. Furthermore, when the user wants to hear a specific sound, the hearing assistance unit can emphasize and filter the sound. As a result, it is possible to provide an optimal hearing experience by filtering the sound based on the current environmental sound.

[0046] The hearing assistance unit can amplify the sound taking into consideration the geographical location information of the user during hearing assistance. For example, the hearing assistance unit can amplify the sound taking into consideration the geographical location information of the user during hearing assistance. The hearing assistance unit provides optimal sound amplification based on the geographical location information of the user. For example, when the user is outdoors, the hearing assistance unit adjusts the sound amplification level according to the surrounding environmental sound. Furthermore, when the user is indoors, the hearing assistance unit can amplify the sound according to the acoustic conditions of the room. Furthermore, when the user is moving, the hearing assistance unit can dynamically adjust the sound amplification based on the geographical location. This allows optimal sound amplification to be performed by taking into consideration the geographical location information.

[0047] The hearing assistance unit can analyze the user's social media activities during hearing assistance and provide related sound information. For example, the hearing assistance unit can analyze the user's social media activities during hearing assistance and provide related sound information. The hearing assistance unit customizes sound amplification based on the user's social media activities. For example, the hearing assistance unit customizes sound amplification based on audio content shared by the user on social media. In addition, the hearing assistance unit can personalize sound amplification based on the content of the user's social media posts. Furthermore, the hearing assistance unit can analyze the user's social media activity history and provide related sound information. In this way, related sound information can be provided by analyzing social media activities.

[0048] The view sharing unit can analyze the user's past view sharing data and select the optimal view sharing method. The view sharing unit can, for example, analyze the user's past view sharing data and select the optimal view sharing method. The view sharing unit selects the most effective view sharing method based on the user's view sharing history and feedback. In addition, the view sharing unit can analyze the user's view sharing pattern and perform view sharing at an appropriate time. For example, the view sharing unit monitors the user's view sharing data over the long term and updates the view sharing method. In this way, the optimal view sharing method can be selected by analyzing the past view sharing data.

[0049] The view sharing unit can adjust the display method of the view information based on the user's current activity content when sharing the view. For example, the view sharing unit can adjust the display method of the view information based on the user's current activity content when sharing the view. The view sharing unit analyzes the user's activity content in real time and provides optimal view information. For example, when the user is reading, the view sharing unit enlarges the text to make it easier to read. In addition, when the user is exercising, the view sharing unit can dynamically adjust the view information according to the movement. Furthermore, when the user is cooking, the view sharing unit can visually highlight the recipe and measurement information. This allows the optimal visual experience to be provided by customizing the display method of the view information based on the user's activity content.

[0050] The view sharing unit can provide optimal view information in consideration of the geographical location information of the user when sharing the view. The view sharing unit can provide optimal view information in consideration of the geographical location information of the user when sharing the view. The view sharing unit provides optimal view information based on the geographical location information of the user. For example, when the user is outdoors, the view sharing unit optimizes the view information by adjusting brightness and contrast. In addition, when the user is indoors, the view sharing unit can adjust the view information according to the lighting conditions. Furthermore, when the user is moving, the view sharing unit can dynamically update the view information based on the geographical location. Thereby, optimal view information can be provided by taking into account the geographical location information.

[0051] The view sharing unit can analyze the user's social media activity during view sharing and provide related view information. The view sharing unit can, for example, analyze the user's social media activity during view sharing and provide related view information. The view sharing unit customizes the view information based on the user's social media activity. For example, the view sharing unit customizes the view information based on photos shared by the user on social media. In addition, the view sharing unit can personalize the view information based on the content posted by the user on social media. Furthermore, the view sharing unit can analyze the user's social media activity history and provide related view information. In this way, related view information can be provided by analyzing social media activity.

[0052] The life management unit can analyze the user's past life data and select the optimal notification method. The life management unit can, for example, analyze the user's past life data and select the optimal notification method. The life management unit selects the most effective notification method based on the user's schedule history and feedback. The life management unit can also analyze the user's life patterns and provide notifications at appropriate times. For example, the life management unit monitors the user's life data over the long term and updates the notification method. In this way, the optimal notification method can be selected by analyzing the past life data.

[0053] The life management unit can adjust the notification content based on the user's current activity content during life management. For example, the life management unit can adjust the notification content based on the user's current activity content during life management. The life management unit analyzes the user's activity content in real time and provides optimal notification. For example, the life management unit selects a quiet notification method when the user is reading. Also, the life management unit can select a visually noticeable notification method when the user is exercising. Furthermore, the life management unit can select a voice notification when the user is cooking. In this way, the notification content can be customized based on the user's activity content to provide optimal notification.

[0054] The life management unit can provide notifications taking into consideration the geographical location information of the user during life management. The life management unit can provide notifications taking into consideration the geographical location information of the user during life management, for example. The life management unit provides optimal notifications based on the geographical location information of the user. For example, when the user is outdoors, the life management unit provides appropriate notifications based on the location information. Furthermore, when the user is indoors, the life management unit can provide notifications tailored to the indoor environment. Furthermore, when the user is moving, the life management unit can dynamically update the notification content based on the geographical location. This allows optimal notifications to be provided by taking into consideration the geographical location information.

[0055] The life management unit can analyze the user's social media activity during life management and provide related notifications. For example, the life management unit can analyze the user's social media activity during life management and provide related notifications. The life management unit customizes notifications based on the user's social media activity. For example, the life management unit customizes notifications based on event information shared by the user on social media. In addition, the life management unit can personalize notifications based on the content of the user's social media posts. Furthermore, the life management unit can analyze the user's social media activity history and provide related notifications. In this way, related notifications can be provided by analyzing social media activity.

[0056] The care assistance unit can analyze the user's past care data and select the optimal care assistance method. The care assistance unit can, for example, analyze the user's past care data and select the optimal care assistance method. The care assistance unit selects the most effective care assistance method based on the user's care history and feedback. In addition, the care assistance unit can analyze the user's care patterns and provide care assistance at an appropriate time. For example, the care assistance unit monitors the user's care data over the long term and updates the care assistance method. In this way, the optimal care assistance method can be selected by analyzing the past care data.

[0057] The care assistance unit can adjust the assistance content based on the user's current physical condition during care assistance. For example, the care assistance unit can adjust the assistance content based on the user's current physical condition during care assistance. The care assistance unit analyzes the user's physical condition in real time and provides optimal care assistance. For example, if the user is tired, the care assistance unit issues a notification encouraging the user to take a break. Also, if the user is exercising, the care assistance unit can suggest an appropriate exercise method. Furthermore, if the user is feeling unwell, the care assistance unit can issue a notification encouraging the user to contact a medical institution. This makes it possible to provide optimal care assistance by customizing the assistance content based on the user's current physical condition.

[0058] The care assistance unit can provide optimal care assistance in consideration of the geographical location information of the user during care assistance. The care assistance unit can provide optimal care assistance in consideration of the geographical location information of the user during care assistance, for example. The care assistance unit provides optimal care assistance based on the geographical location information of the user. For example, when the user is outdoors, the care assistance unit provides appropriate care assistance based on the location information. Furthermore, when the user is indoors, the care assistance unit can provide care assistance that matches the indoor environment. Furthermore, when the user is moving, the care assistance unit can dynamically update the care assistance content based on the geographical location. This allows optimal care assistance to be provided by taking into account the geographical location information.

[0059] The care assistance unit can analyze the user's social media activity during care assistance and provide related care information. The care assistance unit can, for example, analyze the user's social media activity during care assistance and provide related care information. The care assistance unit customizes the care assistance based on the user's social media activity. For example, the care assistance unit customizes the care assistance based on health information shared by the user on social media. In addition, the care assistance unit can personalize the care assistance based on the content of the user's social media posts. Furthermore, the care assistance unit can analyze the user's social media activity history and provide related care information. In this way, related care information can be provided by analyzing social media activity.

[0060] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0061] The vision adjustment unit not only adjusts the user's vision in real time, but also monitors the user's visual fatigue level and can reduce the frequency of vision adjustment when fatigue accumulates. For example, the vision adjustment unit analyzes the user's blinking frequency and eye movement to estimate the visual fatigue level. In addition, the vision adjustment unit can display a notification encouraging the user to take a break when the user continues to look at the screen for a long time. Furthermore, the vision adjustment unit can automatically adjust the brightness and contrast of visual information according to the user's visual fatigue level. This can reduce visual fatigue and provide a comfortable visual experience.

[0062] The hearing aid unit can not only amplify surrounding sounds, but also monitor the user's hearing fatigue level and adjust the sound amplification level if fatigue accumulates. For example, the hearing aid unit can analyze the user's ear movement and reaction speed to estimate the hearing fatigue level. In addition, the hearing aid unit can display a notification encouraging the user to take a break if the user continues to listen to sounds for a long time. Furthermore, the hearing aid unit can automatically adjust the sound amplification level and the intensity of noise canceling according to the user's hearing fatigue level. This can reduce hearing fatigue and provide a comfortable hearing experience.

[0063] The view sharing unit can not only communicate with a remote supporter in real time, but also customize information displayed in the user's field of view. For example, the view sharing unit allows the supporter to edit information displayed in the user's field of view in real time and highlight important information. The view sharing unit can also filter information displayed in the user's field of view and remove unnecessary information. Furthermore, the view sharing unit has a function in which the supporter explains the information displayed in the user's field of view by voice, and can provide support both visually and aurally. This makes it possible to provide more effective support by customizing the information displayed in the user's field of view.

[0064] The lifestyle management unit not only notifies the user when to take medicine, but also monitors the user's health condition and can suggest appropriate measures if an abnormality is detected. For example, the lifestyle management unit monitors the user's body temperature and heart rate in real time, and displays a notification urging the user to contact a medical institution if an abnormality is detected. The lifestyle management unit can also manage the user's diet and exercise records to support healthy lifestyle habits. Furthermore, the lifestyle management unit can suggest appropriate diet and exercise according to the user's health condition. This makes it possible to comprehensively manage the user's health condition and support a healthy lifestyle.

[0065] The care assistant unit can not only display visual guides to compensate for the decline in balance and motor skills, but also analyze the user's motion data and suggest an appropriate rehabilitation program. For example, the care assistant unit can analyze the user's walking data and posture data in real time and provide specific advice on how to maintain balance. The care assistant unit can also suggest rehabilitation programs aimed at improving the user's motor skills and monitor the implementation status. Furthermore, the care assistant unit can accumulate the user's motion data over the long term and analyze changes in motor skills to update the rehabilitation program as appropriate. This can effectively improve the user's motor skills.

[0066] The process flow of the first embodiment will be briefly described below.

[0067] Step 1: The vision adjustment unit adjusts the user's vision in real time. The vision adjustment unit has a focus adjustment function to correct vision, and can enlarge and display visual information using AR technology. For example, the vision adjustment unit automatically adjusts the focus according to the user's vision to display visual information clearly. In addition, the vision adjustment unit has a brightness adjustment function and can automatically adjust the brightness of visual information according to the surrounding light environment. Step 2: The hearing assistance unit amplifies surrounding sounds to assist the user's hearing. The hearing assistance unit amplifies surrounding sounds in decibel units, allowing the user to hear sounds clearly. It also has a noise canceling function to reduce surrounding noise. For example, the hearing assistance unit analyzes surrounding sounds in real time and emphasizes and amplifies important sounds. Step 3: The view sharing unit communicates with the remote supporter in real time and shares the user's view. The view sharing unit uses a high-resolution camera to capture the user's view and transmits it to the remote supporter. It can also share view information in real time using a communication protocol. For example, view information can be transmitted via the Internet, allowing the supporter to give appropriate instructions while viewing the user's view. Step 4: The life management unit supports the user's daily life. The life management unit has a function to notify the user of the time to take medicine, and can manage the user's schedule and display the schedule. For example, the user's schedule is visually displayed using a calendar display or list display. Step 5: The care assistant uses AR technology to display visual guides for maintaining balance to compensate for the decline in balance and motor skills. The care assistant analyzes the user's movements in real time and displays visual guides for maintaining balance. It also has an exercise assistance function and can support the user's motor skills. For example, it monitors the user's movements and suggests appropriate exercise methods.

[0068] (Example 2) The system according to the embodiment of the present invention is an AR smart glass system for assisting the decline in "vision" and "hearing" associated with the aging of the sensory organs of the elderly. This AR smart glass system is equipped with the functions of vision adjustment, hearing assistance, remote assistance, life management, and care assistance, and can compensate for the decline in physical functions associated with the aging of the sensory organs of the elderly. For example, as a vision adjustment function, the AR smart glass adjusts the user's vision in real time. In addition to correcting vision, visual information can be enlarged and displayed using AR technology. Next, as a hearing assistance function, the AR smart glass amplifies surrounding sounds to assist the user's hearing. This allows the user to hear sounds clearly. Furthermore, as a remote assistance function, the AR smart glass communicates with a remote supporter in real time, and the supporter can provide assistance while sharing the user's field of view. For example, a family member or caregiver in a remote location can give appropriate instructions while looking at the user's field of view. In addition, as a life management function, the AR smart glass supports the user's daily life. For example, it can notify the time to take medicine and display a schedule. Finally, as a care assistance function, the AR smart glass assists the user's decline in physical function. For example, AR technology can be used to display visual guides for balancing to compensate for the decline in balance and motor skills, allowing the AR smart glasses system to compensate for the decline in physical functions associated with the aging of the sensory system in the elderly.

[0069] The AR smart glasses system according to the embodiment includes a vision adjustment unit, an auditory assistance unit, a visual field sharing unit, a life management unit, and a care assistance unit. The vision adjustment unit adjusts the user's visual acuity in real time. The vision adjustment unit includes, for example, a focus adjustment function to correct the visual acuity. The vision adjustment unit can also enlarge and display visual information using AR technology. For example, the vision adjustment unit automatically adjusts the focus according to the user's visual acuity to display the visual information clearly. Furthermore, the vision adjustment unit includes a brightness adjustment function and can automatically adjust the brightness of the visual information according to the surrounding light environment. The auditory assistance unit amplifies surrounding sounds and assists the user's hearing. For example, the auditory assistance unit amplifies surrounding sounds in decibel units to enable the user to hear sounds clearly. Furthermore, the auditory assistance unit includes a noise canceling function and can reduce surrounding noise. For example, the auditory assistance unit analyzes surrounding sounds in real time and emphasizes and amplifies important sounds. The visual field sharing unit communicates with a remote supporter in real time and shares the user's visual field. The view sharing unit, for example, photographs the user's view using a high-resolution camera and transmits it to a supporter in a remote location. The view sharing unit can also share view information in real time using a communication protocol. For example, the view sharing unit transmits view information via the Internet, and the supporter can give appropriate instructions while viewing the user's view. The life management unit supports the user's daily life. The life management unit, for example, has a function of notifying the user of the time to take medicine. The life management unit can also manage the user's schedule and display the plan. For example, the life management unit visually displays the user's plan using a calendar display or a list display. The care support unit displays a visual guide for maintaining balance using AR technology to compensate for a decrease in balance and motor skills. The care support unit, for example, analyzes the user's movements in real time and displays a visual guide for maintaining balance. The care support unit also has an exercise support function and can support the user's motor skills. For example, the care support unit monitors the user's movements and suggests an appropriate exercise method. As a result, the AR smart glasses system according to the embodiment can compensate for the decline in physical functions that accompanies the aging of the sensory organs in the elderly.

[0070] The vision adjustment unit adjusts the user's vision in real time. The vision adjustment unit has, for example, a focus adjustment function to correct the vision. Specifically, the vision adjustment unit employs active lens technology to automatically adjust the focus according to the user's vision. This active lens can instantly change the focal length using liquid crystal or electrically controlled materials. The vision adjustment unit can also enlarge and display visual information using AR technology. For example, when a user reads small characters, the vision adjustment unit automatically enlarges the characters to make them easier to read. Furthermore, the vision adjustment unit has a brightness adjustment function and can automatically adjust the brightness of visual information according to the surrounding light environment. This allows the user to always check information in an optimal visual environment, regardless of day or night. The vision adjustment unit continuously monitors the user's vision data and makes adjustments as necessary, thereby reducing eye fatigue even with long-term use. Furthermore, the vision adjustment unit can periodically perform a vision test according to changes in the user's vision and provide optimal correction based on the results. This allows the vision adjustment unit to always keep the user's vision in an optimal state and provide a comfortable visual experience.

[0071] The hearing assistance unit amplifies surrounding sounds and assists the user's hearing. For example, the hearing assistance unit amplifies surrounding sounds in decibel units, allowing the user to hear the sounds clearly. Specifically, the hearing assistance unit collects surrounding sounds using a highly sensitive microphone and amplifies the sounds using digital signal processing technology. The hearing assistance unit also has a noise canceling function and can reduce surrounding noise. For example, the hearing assistance unit analyzes surrounding sounds in real time and emphasizes and amplifies important sounds. This allows the user to clearly hear necessary audio information even in a noisy environment. Furthermore, the hearing assistance unit can adjust the frequency characteristics of the sound according to the user's hearing characteristics. For example, for a user who has difficulty hearing high-pitched sounds, the high-pitched sounds are emphasized and amplified to optimize the overall sound balance. The hearing assistance unit continuously monitors the user's hearing data and makes adjustments as necessary, thereby reducing ear fatigue even with long-term use. Furthermore, the hearing assistance unit can periodically perform hearing tests according to changes in the user's hearing and provide optimal assistance based on the results. This allows the hearing assistance unit to always keep the user's hearing in an optimal state, providing a comfortable hearing experience.

[0072] The view sharing unit communicates with a remote supporter in real time and shares the user's view. The view sharing unit, for example, uses a high-resolution camera to capture the user's view and transmits it to the remote supporter. Specifically, the view sharing unit captures the user's view with a wide-angle lens, compresses the video, and transmits it via the Internet. The view sharing unit can also share view information in real time using a communication protocol. For example, the view sharing unit can use low-latency streaming technology to enable the supporter to check the user's view in almost real time. This allows the supporter to give appropriate instructions while viewing the user's view. Furthermore, the view sharing unit has a two-way communication function, and the supporter and the user can communicate by voice or text. This allows the supporter to provide specific instructions and advice to the user in real time. The view sharing unit encrypts and transmits the user's view information, so privacy protection is also ensured. This allows the view sharing unit to realize safe and efficient communication between the supporter and the user in a remote location and support the user's life.

[0073] The life management unit supports the user's daily life. The life management unit has, for example, a function of notifying the user of the time to take medicine. Specifically, the life management unit can manage the user's schedule and display the plan. For example, the life management unit visually displays the user's plan using a calendar display or a list display. Furthermore, the life management unit can monitor the user's health condition and provide health management advice as necessary. For example, the life management unit measures the user's heart rate and blood pressure, and issues an alert when an abnormality is detected. In addition, the life management unit can manage the user's diet and exercise records and make suggestions to support a balanced life. In this way, the life management unit can comprehensively support the user's daily life and realize a healthy and comfortable life. In addition, the life management unit can provide notifications and reminders customized based on the user's preferences and habits. For example, the life management unit provides support tailored to the user's lifestyle, such as a reminder to take medicine at a specific time and a suggestion for regular exercise. In this way, the life management unit can improve the user's quality of life and make daily life more convenient and comfortable.

[0074] The care assistant unit displays visual guides for maintaining balance using AR technology to compensate for the decline in balance and motor skills. Specifically, the care assistant unit analyzes the user's movements in real time and displays visual guides for maintaining balance. For example, if the user is about to lose balance while walking, the care assistant unit displays visual lines and arrows to provide instructions for the user to regain balance. The care assistant unit also has an exercise assistance function and can support the user's motor skills. For example, the care assistant unit monitors the user's movements and suggests appropriate exercise methods. This allows the user to safely exercise and maintain and improve their motor skills. Furthermore, the care assistant unit can accumulate the user's motion data and analyze long-term changes in motor skills. This allows the care assistant unit to detect the decline in the user's motor skills early and take appropriate measures. The care assistant unit provides important functions for supporting independence in daily life while ensuring the safety of the user. This allows the care assistant unit to improve the quality of life of elderly people and users with reduced motor skills, allowing them to live their daily lives with peace of mind.

[0075] The vision adjustment unit can enlarge and display visual information using AR technology. The vision adjustment unit, for example, uses AR technology to enlarge and display visual information. The AR technology can enlarge the user's visual information using an augmented reality display method. For example, the vision adjustment unit automatically enlarges visual information according to the user's visual acuity to reduce visual burden. In addition, the vision adjustment unit can optimize visual information according to the device used. For example, the vision adjustment unit uses a display of smart glasses to enlarge and display visual information. This makes it possible to compensate for a decrease in visual acuity by enlarging and displaying visual information.

[0076] The hearing assistance unit amplifies surrounding sounds and allows the user to hear the sounds clearly. For example, the hearing assistance unit amplifies surrounding sounds and allows the user to hear the sounds clearly. The hearing assistance unit amplifies sounds in decibel units to assist the user's hearing. The hearing assistance unit can also reduce surrounding noise using noise reduction technology. For example, the hearing assistance unit analyzes surrounding sounds in real time and emphasizes and amplifies important sounds. This makes it possible to compensate for a decrease in hearing by amplifying surrounding sounds.

[0077] The view sharing unit communicates with a remote supporter in real time, and the supporter can give instructions while viewing the user's view. The view sharing unit communicates with a remote supporter in real time, and the supporter can give instructions while viewing the user's view. The view sharing unit captures the user's view using a high-resolution camera and transmits it to the remote supporter. The view sharing unit can also share view information in real time using a communication protocol. For example, the view sharing unit transmits view information via the Internet, and the supporter can give appropriate instructions while viewing the user's view. This allows the remote supporter to provide appropriate support by sharing the user's view.

[0078] The life management unit can notify the user of the time to take medicine and display a schedule. The life management unit can, for example, notify the user of the time to take medicine and display a schedule. The life management unit notifies the user of the time to take medicine using an alarm sound or vibration. The life management unit can also visually display the user's schedule using a calendar display or a list display. For example, the life management unit manages the user's schedule and notifies the user at an appropriate time. In this way, it is possible to support daily life by notifying the user of the time to take medicine and displaying the schedule.

[0079] The care assistant unit can display a visual guide for maintaining balance using AR technology. The care assistant unit can display a visual guide for maintaining balance using AR technology, for example. The care assistant unit displays a visual guide for maintaining balance for the user using an arrow display or a line display. In addition, the care assistant unit can analyze the user's movements in real time and display appropriate guides. For example, the care assistant unit monitors the user's movements and displays a visual guide for maintaining balance. In this way, a decline in balance and motor skills can be compensated for by displaying a visual guide for maintaining balance using AR technology.

[0080] The vision adjustment unit can estimate the user's emotion and adjust the frequency of the vision adjustment based on the estimated user's emotion. The vision adjustment unit can, for example, estimate the user's emotion and adjust the frequency of the vision adjustment based on the estimated user's emotion. The vision adjustment unit estimates the user's emotion using an emotion estimation algorithm. For example, the vision adjustment unit uses facial expression recognition technology to analyze the user's facial expression and estimate the emotion. The vision adjustment unit can also use voice analysis technology to analyze the tone and speed of the user's voice and estimate the emotion. For example, the vision adjustment unit reduces the frequency of the vision adjustment when the user is stressed, thereby reducing the visual burden. Also, the vision adjustment unit can increase the frequency of the vision adjustment when the user is relaxed, thereby providing an optimal visual experience. Furthermore, when the user is concentrating, the vision adjustment unit keeps the frequency of the vision adjustment moderate so as not to interfere with the work. As a result, the visual burden can be reduced by adjusting the frequency of the vision adjustment based on the user's emotion. The emotion estimation is realized using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples.

[0081] The vision adjustment unit can analyze the user's past vision data and select the optimal vision adjustment method. The vision adjustment unit can, for example, analyze the user's past vision data and select the optimal vision adjustment method. The vision adjustment unit selects the most effective vision correction method based on the user's vision test results and feedback. In addition, the vision adjustment unit can analyze the user's vision change pattern and adjust the vision at an appropriate time. For example, the vision adjustment unit monitors the user's vision data over the long term and updates the adjustment method according to changes in vision. In this way, the optimal vision adjustment method can be selected by analyzing past vision data.

[0082] The vision adjustment unit can adjust the display method of visual information based on the user's current activity content during vision adjustment. For example, the vision adjustment unit can adjust the display method of visual information based on the user's current activity content during vision adjustment. The vision adjustment unit analyzes the user's activity content in real time to provide optimal visual information. For example, when the user is reading, the vision adjustment unit enlarges and displays characters to make them easier to read. In addition, when the user is exercising, the vision adjustment unit can dynamically adjust the visual information according to the movement. Furthermore, when the user is cooking, the vision adjustment unit can visually highlight recipes and measurement information. This makes it possible to provide an optimal visual experience by customizing the display method of visual information based on the user's activity content.

[0083] The vision adjustment unit can estimate the user's emotion and determine the priority of the vision adjustment based on the estimated user's emotion. The vision adjustment unit can, for example, estimate the user's emotion and determine the priority of the vision adjustment based on the estimated user's emotion. The vision adjustment unit estimates the user's emotion using an emotion estimation algorithm. For example, the vision adjustment unit uses facial expression recognition technology to analyze the user's facial expression and estimate the emotion. The vision adjustment unit can also use voice analysis technology to analyze the tone and speed of the user's voice and estimate the emotion. For example, when the user feels anxious, the vision adjustment unit prioritizes the vision adjustment to provide a visual sense of security. When the user is relaxed, the vision adjustment unit can lower the priority of the vision adjustment to maintain a natural visual experience. Furthermore, when the user is concentrating, the vision adjustment unit sets the priority of the vision adjustment to a medium level so as not to interfere with the work. In this way, a visual sense of security can be provided by determining the priority of the vision adjustment based on the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples.

[0084] The vision adjustment unit can provide appropriate visual information in consideration of the geographical location information of the user when adjusting the vision. The vision adjustment unit can provide appropriate visual information in consideration of the geographical location information of the user when adjusting the vision. The vision adjustment unit provides optimal visual information based on the geographical location information of the user. For example, when the user is outdoors, the vision adjustment unit adjusts brightness and contrast to optimize the visual information. In addition, when the user is indoors, the vision adjustment unit can adjust the visual information according to the lighting conditions. Furthermore, when the user is moving, the vision adjustment unit can dynamically update the visual information based on the geographical location. Thereby, optimal visual information can be provided by taking into account the geographical location information.

[0085] The vision adjustment unit can analyze the user's social media activity and provide related visual information when adjusting the vision. For example, the vision adjustment unit can analyze the user's social media activity and provide related visual information when adjusting the vision. The vision adjustment unit customizes the visual information based on the user's social media activity. For example, the vision adjustment unit customizes the visual information based on photos shared by the user on social media. In addition, the vision adjustment unit can personalize the visual information based on the content of the user's social media posts. Furthermore, the vision adjustment unit can analyze the user's social media activity history and provide related visual information. In this way, it is possible to provide related visual information by analyzing social media activity.

[0086] The hearing assistance unit can estimate the user's emotion and adjust the sound amplification level based on the estimated user's emotion. The hearing assistance unit can, for example, estimate the user's emotion and adjust the sound amplification level based on the estimated user's emotion. The hearing assistance unit estimates the user's emotion using an emotion estimation algorithm. For example, the hearing assistance unit uses facial expression recognition technology to analyze the user's facial expression and estimate the emotion. The hearing assistance unit can also use voice analysis technology to analyze the tone and speed of the user's voice and estimate the emotion. For example, when the user is feeling stressed, the hearing assistance unit can provide an environment in which the user can relax by lowering the sound amplification level. In addition, when the user is relaxed, the hearing assistance unit can maintain the sound amplification level at a moderate level to provide a comfortable hearing experience. Furthermore, when the user is concentrating, the hearing assistance unit can adjust the sound amplification level to block out external noise. As a result, a comfortable hearing experience can be provided by adjusting the sound amplification level based on the user's emotion. The emotion estimation is realized using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples.

[0087] The hearing assistance unit can analyze the user's past hearing data and select the optimal sound amplification method. The hearing assistance unit can, for example, analyze the user's past hearing data and select the optimal sound amplification method. The hearing assistance unit selects the most effective sound amplification method based on the user's hearing test results and feedback. In addition, the hearing assistance unit can analyze the user's hearing fluctuation pattern and amplify the sound at an appropriate time. For example, the hearing assistance unit monitors the user's hearing data over the long term and updates the amplification method according to changes in hearing. In this way, the optimal sound amplification method can be selected by analyzing the past hearing data.

[0088] The hearing assistance unit can filter the sound based on the user's current environmental sound during hearing assistance. For example, the hearing assistance unit can filter the sound based on the user's current environmental sound during hearing assistance. The hearing assistance unit analyzes the surrounding noise level in real time and performs optimal sound filtering. For example, when the user is in a noisy environment, the hearing assistance unit activates a noise canceling function to block external noise. In addition, when the user is in a quiet environment, the hearing assistance unit can naturally amplify the surrounding sound. Furthermore, when the user wants to hear a specific sound, the hearing assistance unit can emphasize and filter the sound. As a result, it is possible to provide an optimal hearing experience by filtering the sound based on the current environmental sound.

[0089] The hearing assistance unit can estimate the user's emotion and determine the amplification priority of the sound based on the estimated user's emotion. The hearing assistance unit can, for example, estimate the user's emotion and determine the amplification priority of the sound based on the estimated user's emotion. The hearing assistance unit estimates the user's emotion using an emotion estimation algorithm. For example, the hearing assistance unit uses facial expression recognition technology to analyze the user's facial expression and estimate the emotion. The hearing assistance unit can also use voice analysis technology to analyze the tone and speed of the user's voice and estimate the emotion. For example, when the user feels anxious, the hearing assistance unit preferentially amplifies important sounds to provide a sense of security. When the user is relaxed, the hearing assistance unit can lower the amplification priority of the sound to provide a natural hearing experience. Furthermore, when the user is concentrating, the hearing assistance unit can block out external noise and emphasize important sounds. As a result, important sounds can be preferentially amplified by determining the amplification priority based on the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples.

[0090] The hearing assistance unit can amplify the sound taking into consideration the geographical location information of the user during hearing assistance. For example, the hearing assistance unit can amplify the sound taking into consideration the geographical location information of the user during hearing assistance. The hearing assistance unit provides optimal sound amplification based on the geographical location information of the user. For example, when the user is outdoors, the hearing assistance unit adjusts the sound amplification level according to the surrounding environmental sound. Furthermore, when the user is indoors, the hearing assistance unit can amplify the sound according to the acoustic conditions of the room. Furthermore, when the user is moving, the hearing assistance unit can dynamically adjust the sound amplification based on the geographical location. This allows optimal sound amplification to be performed by taking into consideration the geographical location information.

[0091] The hearing assistance unit can analyze the user's social media activities during hearing assistance and provide related sound information. For example, the hearing assistance unit can analyze the user's social media activities during hearing assistance and provide related sound information. The hearing assistance unit customizes sound amplification based on the user's social media activities. For example, the hearing assistance unit customizes sound amplification based on audio content shared by the user on social media. In addition, the hearing assistance unit can personalize sound amplification based on the content of the user's social media posts. Furthermore, the hearing assistance unit can analyze the user's social media activity history and provide related sound information. In this way, related sound information can be provided by analyzing social media activities.

[0092] The view sharing unit can estimate the user's emotion and adjust the frequency of view sharing based on the estimated user's emotion. The view sharing unit can, for example, estimate the user's emotion and adjust the frequency of view sharing based on the estimated user's emotion. The view sharing unit estimates the user's emotion using an emotion estimation algorithm. For example, the view sharing unit uses facial expression recognition technology to analyze the user's facial expression and estimate the emotion. The view sharing unit can also use voice analysis technology to analyze the tone and speed of the user's voice and estimate the emotion. For example, when the user feels anxious, the view sharing unit can increase the frequency of view sharing to provide a sense of security. When the user is relaxed, the view sharing unit can reduce the frequency of view sharing to provide a natural experience. Furthermore, when the user is concentrating, the view sharing unit keeps the frequency of view sharing moderate so as not to interfere with the work. This makes it possible to provide a sense of security by adjusting the frequency of view sharing based on the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples.

[0093] The view sharing unit can analyze the user's past view sharing data and select the optimal view sharing method. The view sharing unit can, for example, analyze the user's past view sharing data and select the optimal view sharing method. The view sharing unit selects the most effective view sharing method based on the user's view sharing history and feedback. In addition, the view sharing unit can analyze the user's view sharing pattern and perform view sharing at an appropriate time. For example, the view sharing unit monitors the user's view sharing data over the long term and updates the view sharing method. In this way, the optimal view sharing method can be selected by analyzing the past view sharing data.

[0094] The view sharing unit can adjust the display method of the view information based on the user's current activity content when sharing the view. For example, the view sharing unit can adjust the display method of the view information based on the user's current activity content when sharing the view. The view sharing unit analyzes the user's activity content in real time and provides optimal view information. For example, when the user is reading, the view sharing unit enlarges the text to make it easier to read. In addition, when the user is exercising, the view sharing unit can dynamically adjust the view information according to the movement. Furthermore, when the user is cooking, the view sharing unit can visually highlight the recipe and measurement information. This allows the optimal visual experience to be provided by customizing the display method of the view information based on the user's activity content.

[0095] The view sharing unit can estimate the user's emotion and determine the priority of view sharing based on the estimated user's emotion. The view sharing unit can, for example, estimate the user's emotion and determine the priority of view sharing based on the estimated user's emotion. The view sharing unit estimates the user's emotion using an emotion estimation algorithm. For example, the view sharing unit uses facial expression recognition technology to analyze the user's facial expression and estimate the emotion. The view sharing unit can also use voice analysis technology to analyze the tone and speed of the user's voice and estimate the emotion. For example, when the user feels anxious, the view sharing unit prioritizes view sharing to provide a sense of security. In addition, when the user is relaxed, the view sharing unit can lower the priority of view sharing to provide a natural experience. Furthermore, when the user is concentrating, the view sharing unit sets the priority of view sharing to a medium level so as not to interfere with the work. In this way, a sense of security can be provided by determining the priority of view sharing based on the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples.

[0096] The view sharing unit can provide optimal view information in consideration of the geographical location information of the user when sharing the view. The view sharing unit can provide optimal view information in consideration of the geographical location information of the user when sharing the view. The view sharing unit provides optimal view information based on the geographical location information of the user. For example, when the user is outdoors, the view sharing unit optimizes the view information by adjusting brightness and contrast. In addition, when the user is indoors, the view sharing unit can adjust the view information according to the lighting conditions. Furthermore, when the user is moving, the view sharing unit can dynamically update the view information based on the geographical location. Thereby, optimal view information can be provided by taking into account the geographical location information.

[0097] The view sharing unit can analyze the user's social media activity during view sharing and provide related view information. The view sharing unit can, for example, analyze the user's social media activity during view sharing and provide related view information. The view sharing unit customizes the view information based on the user's social media activity. For example, the view sharing unit customizes the view information based on photos shared by the user on social media. In addition, the view sharing unit can personalize the view information based on the content posted by the user on social media. Furthermore, the view sharing unit can analyze the user's social media activity history and provide related view information. In this way, related view information can be provided by analyzing social media activity.

[0098] The life management unit can estimate the user's emotion and adjust the frequency of notifications based on the estimated user's emotion. The life management unit can, for example, estimate the user's emotion and adjust the frequency of notifications based on the estimated user's emotion. The life management unit estimates the user's emotion using an emotion estimation algorithm. For example, the life management unit uses facial expression recognition technology to analyze the user's facial expression and estimate the emotion. The life management unit can also use voice analysis technology to analyze the tone and speed of the user's voice and estimate the emotion. For example, when the user is feeling stressed, the life management unit reduces the frequency of notifications and provides an environment in which the user can relax. In addition, when the user is relaxed, the life management unit can keep the frequency of notifications moderate and provide necessary information. Furthermore, when the user is concentrating, the life management unit adjusts the frequency of notifications so as not to interfere with the work. In this way, by adjusting the frequency of notifications based on the user's emotion, an environment in which the user can relax can be provided. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples.

[0099] The life management unit can analyze the user's past life data and select the optimal notification method. The life management unit can, for example, analyze the user's past life data and select the optimal notification method. The life management unit selects the most effective notification method based on the user's schedule history and feedback. The life management unit can also analyze the user's life patterns and provide notifications at appropriate times. For example, the life management unit monitors the user's life data over the long term and updates the notification method. In this way, the optimal notification method can be selected by analyzing the past life data.

[0100] The life management unit can adjust the notification content based on the user's current activity content during life management. For example, the life management unit can adjust the notification content based on the user's current activity content during life management. The life management unit analyzes the user's activity content in real time and provides optimal notification. For example, the life management unit selects a quiet notification method when the user is reading. Also, the life management unit can select a visually noticeable notification method when the user is exercising. Furthermore, the life management unit can select a voice notification when the user is cooking. In this way, the notification content can be customized based on the user's activity content to provide optimal notification.

[0101] The life management unit can estimate the user's emotion and determine the priority of notifications based on the estimated user's emotion. The life management unit can, for example, estimate the user's emotion and determine the priority of notifications based on the estimated user's emotion. The life management unit estimates the user's emotion using an emotion estimation algorithm. For example, the life management unit uses facial expression recognition technology to analyze the user's facial expression and estimate the emotion. The life management unit can also use voice analysis technology to analyze the tone and speed of the user's voice and estimate the emotion. For example, when the user feels anxious, the life management unit prioritizes important notifications to provide a sense of security. In addition, when the user is relaxed, the life management unit can lower the priority of notifications to provide a natural experience. Furthermore, when the user is concentrating, the life management unit prioritizes important notifications and postpones other notifications. In this way, a sense of security can be provided by determining the priority of notifications based on the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples.

[0102] The life management unit can provide notifications taking into consideration the geographical location information of the user during life management. The life management unit can provide notifications taking into consideration the geographical location information of the user during life management, for example. The life management unit provides optimal notifications based on the geographical location information of the user. For example, when the user is outdoors, the life management unit provides appropriate notifications based on the location information. Furthermore, when the user is indoors, the life management unit can provide notifications tailored to the indoor environment. Furthermore, when the user is moving, the life management unit can dynamically update the notification content based on the geographical location. This allows optimal notifications to be provided by taking into consideration the geographical location information.

[0103] The life management unit can analyze the user's social media activity during life management and provide related notifications. For example, the life management unit can analyze the user's social media activity during life management and provide related notifications. The life management unit customizes notifications based on the user's social media activity. For example, the life management unit customizes notifications based on event information shared by the user on social media. In addition, the life management unit can personalize notifications based on the content of the user's social media posts. Furthermore, the life management unit can analyze the user's social media activity history and provide related notifications. In this way, related notifications can be provided by analyzing social media activity.

[0104] The care assistance unit can estimate the user's emotion and adjust the frequency of care assistance based on the estimated user's emotion. The care assistance unit can, for example, estimate the user's emotion and adjust the frequency of care assistance based on the estimated user's emotion. The care assistance unit estimates the user's emotion using an emotion estimation algorithm. For example, the care assistance unit uses facial expression recognition technology to analyze the user's facial expression and estimate the emotion. The care assistance unit can also use voice analysis technology to analyze the tone and speed of the user's voice and estimate the emotion. For example, when the user feels anxious, the care assistance unit can increase the frequency of care assistance to provide a sense of security. When the user is relaxed, the care assistance unit can reduce the frequency of care assistance to provide a natural experience. Furthermore, when the user is concentrating, the care assistance unit keeps the frequency of care assistance moderate so as not to interfere with the work. This makes it possible to provide a sense of security by adjusting the frequency of care assistance based on the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples.

[0105] The care assistance unit can analyze the user's past care data and select the optimal care assistance method. The care assistance unit can, for example, analyze the user's past care data and select the optimal care assistance method. The care assistance unit selects the most effective care assistance method based on the user's care history and feedback. In addition, the care assistance unit can analyze the user's care patterns and provide care assistance at an appropriate time. For example, the care assistance unit monitors the user's care data over the long term and updates the care assistance method. In this way, the optimal care assistance method can be selected by analyzing the past care data.

[0106] The care assistance unit can adjust the assistance content based on the user's current physical condition during care assistance. For example, the care assistance unit can adjust the assistance content based on the user's current physical condition during care assistance. The care assistance unit analyzes the user's physical condition in real time and provides optimal care assistance. For example, if the user is tired, the care assistance unit issues a notification encouraging the user to take a break. Also, if the user is exercising, the care assistance unit can suggest an appropriate exercise method. Furthermore, if the user is feeling unwell, the care assistance unit can issue a notification encouraging the user to contact a medical institution. This makes it possible to provide optimal care assistance by customizing the assistance content based on the user's current physical condition.

[0107] The care assistance unit can estimate the user's emotion and determine the priority of care assistance based on the estimated user's emotion. The care assistance unit can, for example, estimate the user's emotion and determine the priority of care assistance based on the estimated user's emotion. The care assistance unit estimates the user's emotion using an emotion estimation algorithm. For example, the care assistance unit uses facial expression recognition technology to analyze the user's facial expression and estimate the emotion. The care assistance unit can also use voice analysis technology to analyze the tone and speed of the user's voice and estimate the emotion. For example, when the user feels anxious, the care assistance unit prioritizes care assistance and provides a sense of security. In addition, when the user is relaxed, the care assistance unit can lower the priority of care assistance and provide a natural experience. Furthermore, when the user is concentrating, the care assistance unit sets the priority of care assistance to a medium level so as not to interfere with the work. In this way, a sense of security can be provided by determining the priority of care assistance based on the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples.

[0108] The care assistance unit can provide optimal care assistance in consideration of the geographical location information of the user during care assistance. The care assistance unit can provide optimal care assistance in consideration of the geographical location information of the user during care assistance, for example. The care assistance unit provides optimal care assistance based on the geographical location information of the user. For example, when the user is outdoors, the care assistance unit provides appropriate care assistance based on the location information. Furthermore, when the user is indoors, the care assistance unit can provide care assistance that matches the indoor environment. Furthermore, when the user is moving, the care assistance unit can dynamically update the care assistance content based on the geographical location. This allows optimal care assistance to be provided by taking into account the geographical location information.

[0109] The care assistance unit can analyze the user's social media activity during care assistance and provide related care information. The care assistance unit can, for example, analyze the user's social media activity during care assistance and provide related care information. The care assistance unit customizes the care assistance based on the user's social media activity. For example, the care assistance unit customizes the care assistance based on health information shared by the user on social media. In addition, the care assistance unit can personalize the care assistance based on the content of the user's social media posts. Furthermore, the care assistance unit can analyze the user's social media activity history and provide related care information. In this way, related care information can be provided by analyzing social media activity.

[0110] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0111] The vision adjustment unit not only adjusts the user's vision in real time, but also monitors the user's visual fatigue level and can reduce the frequency of vision adjustment when fatigue accumulates. For example, the vision adjustment unit analyzes the user's blinking frequency and eye movement to estimate the visual fatigue level. In addition, the vision adjustment unit can display a notification encouraging the user to take a break when the user continues to look at the screen for a long time. Furthermore, the vision adjustment unit can automatically adjust the brightness and contrast of visual information according to the user's visual fatigue level. This can reduce visual fatigue and provide a comfortable visual experience.

[0112] The hearing aid unit can not only amplify surrounding sounds, but also monitor the user's hearing fatigue level and adjust the sound amplification level if fatigue accumulates. For example, the hearing aid unit can analyze the user's ear movement and reaction speed to estimate the hearing fatigue level. In addition, the hearing aid unit can display a notification encouraging the user to take a break if the user continues to listen to sounds for a long time. Furthermore, the hearing aid unit can automatically adjust the sound amplification level and the intensity of noise canceling according to the user's hearing fatigue level. This can reduce hearing fatigue and provide a comfortable hearing experience.

[0113] The view sharing unit can not only communicate with a remote supporter in real time, but also customize information displayed in the user's field of view. For example, the view sharing unit allows the supporter to edit information displayed in the user's field of view in real time and highlight important information. The view sharing unit can also filter information displayed in the user's field of view and remove unnecessary information. Furthermore, the view sharing unit has a function in which the supporter explains the information displayed in the user's field of view by voice, and can provide support both visually and aurally. This makes it possible to provide more effective support by customizing the information displayed in the user's field of view.

[0114] The lifestyle management unit not only notifies the user when to take medicine, but also monitors the user's health condition and can suggest appropriate measures if an abnormality is detected. For example, the lifestyle management unit monitors the user's body temperature and heart rate in real time, and displays a notification urging the user to contact a medical institution if an abnormality is detected. The lifestyle management unit can also manage the user's diet and exercise records to support healthy lifestyle habits. Furthermore, the lifestyle management unit can suggest appropriate diet and exercise according to the user's health condition. This makes it possible to comprehensively manage the user's health condition and support a healthy lifestyle.

[0115] The care assistant unit can not only display visual guides to compensate for the decline in balance and motor skills, but also analyze the user's motion data and suggest an appropriate rehabilitation program. For example, the care assistant unit can analyze the user's walking data and posture data in real time and provide specific advice on how to maintain balance. The care assistant unit can also suggest rehabilitation programs aimed at improving the user's motor skills and monitor the implementation status. Furthermore, the care assistant unit can accumulate the user's motion data over the long term and analyze changes in motor skills to update the rehabilitation program as appropriate. This can effectively improve the user's motor skills.

[0116] The vision adjustment unit can estimate the user's emotions and adjust the frequency of vision adjustment based on the estimated user's emotions. For example, when the user is stressed, the vision adjustment unit can reduce the frequency of vision adjustment to reduce visual strain. Also, when the user is relaxed, the vision adjustment unit can increase the frequency of vision adjustment to provide an optimal visual experience. Furthermore, when the user is concentrating, the vision adjustment unit can keep the frequency of vision adjustment moderate so as not to interfere with work. In this way, the visual strain can be reduced by adjusting the frequency of vision adjustment based on the user's emotions.

[0117] The hearing assistance unit can estimate the user's emotions and adjust the sound amplification level based on the estimated user's emotions. For example, when the user feels stressed, the hearing assistance unit can lower the sound amplification level to provide a relaxing environment. When the user is relaxed, the hearing assistance unit can maintain a moderate sound amplification level to provide a comfortable hearing experience. Furthermore, when the user is concentrating, the hearing assistance unit can adjust the sound amplification level to block out external noise. This allows a comfortable hearing experience to be provided by adjusting the sound amplification level based on the user's emotions.

[0118] The view sharing unit can estimate the user's emotions and adjust the frequency of view sharing based on the estimated user's emotions. For example, when the user feels anxious, the view sharing unit can increase the frequency of view sharing to provide a sense of security. Also, when the user is relaxed, the view sharing unit can reduce the frequency of view sharing to provide a natural experience. Furthermore, when the user is concentrating, the view sharing unit can keep the frequency of view sharing moderate so as not to interfere with the work. In this way, a sense of security can be provided by adjusting the frequency of view sharing based on the user's emotions.

[0119] The life management unit can estimate the user's emotions and adjust the frequency of notifications based on the estimated user's emotions. For example, when the user is feeling stressed, the life management unit can reduce the frequency of notifications and provide an environment in which the user can relax. Also, when the user is relaxed, the life management unit can keep the frequency of notifications moderate and provide necessary information. Furthermore, when the user is concentrating, the life management unit adjusts the frequency of notifications so as not to interfere with the user's work. In this way, by adjusting the frequency of notifications based on the user's emotions, an environment in which the user can relax can be provided.

[0120] The care assistance unit can estimate the user's emotions and adjust the frequency of care assistance based on the estimated user's emotions. For example, when the user feels anxious, the care assistance unit can increase the frequency of care assistance to provide a sense of security. Also, when the user is relaxed, the care assistance unit can reduce the frequency of care assistance to provide a natural experience. Furthermore, when the user is concentrating, the care assistance unit can keep the frequency of care assistance moderate so as not to interfere with the work. In this way, a sense of security can be provided by adjusting the frequency of care assistance based on the user's emotions.

[0121] The process flow of the second embodiment will be briefly described below.

[0122] Step 1: The vision adjustment unit adjusts the user's vision in real time. The vision adjustment unit has a focus adjustment function to correct vision, and can enlarge and display visual information using AR technology. For example, the vision adjustment unit automatically adjusts the focus according to the user's vision to display visual information clearly. In addition, the vision adjustment unit has a brightness adjustment function and can automatically adjust the brightness of visual information according to the surrounding light environment. Step 2: The hearing assistance unit amplifies surrounding sounds to assist the user's hearing. The hearing assistance unit amplifies surrounding sounds in decibel units, allowing the user to hear sounds clearly. It also has a noise canceling function to reduce surrounding noise. For example, the hearing assistance unit analyzes surrounding sounds in real time and emphasizes and amplifies important sounds. Step 3: The view sharing unit communicates with the remote supporter in real time and shares the user's view. The view sharing unit uses a high-resolution camera to capture the user's view and transmits it to the remote supporter. It can also share view information in real time using a communication protocol. For example, view information can be transmitted via the Internet, allowing the supporter to give appropriate instructions while viewing the user's view. Step 4: The life management unit supports the user's daily life. The life management unit has a function to notify the user of the time to take medicine, and can manage the user's schedule and display the schedule. For example, the user's schedule is visually displayed using a calendar display or list display. Step 5: The care assistant uses AR technology to display visual guides for maintaining balance to compensate for the decline in balance and motor skills. The care assistant analyzes the user's movements in real time and displays visual guides for maintaining balance. It also has an exercise assistance function and can support the user's motor skills. For example, it monitors the user's movements and suggests appropriate exercise methods.

[0123] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0124] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0125] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0126] Each of the multiple elements including the vision adjustment unit, the hearing assistance unit, the view sharing unit, the life management unit, and the care assistance unit described above is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the vision adjustment unit is realized by the control unit 46A of the smart device 14 and adjusts the user's vision in real time. The hearing assistance unit is realized by the control unit 46A of the smart device 14 and amplifies surrounding sounds to assist the user's hearing. The view sharing unit photographs the user's field of vision using the camera 42 of the smart device 14 and transmits it to a remote supporter via the specific processing unit 290 of the data processing device 12. The life management unit is realized by the control unit 46A of the smart device 14 and supports the user's daily life. The care assistance unit is realized by the control unit 46A of the smart device 14 and displays a visual guide for balancing using AR technology. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0127] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0128] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0129] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0130] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0131] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0132] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0133] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0134] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0135] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0136] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0137] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0138] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0139] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0140] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0141] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0142] Each of the multiple elements including the vision adjustment unit, the hearing assistance unit, the field of view sharing unit, the life management unit, and the care assistance unit described above is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the vision adjustment unit is realized by the control unit 46A of the smart glasses 214 and adjusts the user's vision in real time. The hearing assistance unit is realized by the control unit 46A of the smart glasses 214 and amplifies surrounding sounds to assist the user's hearing. The field of view sharing unit photographs the user's field of view using the camera 42 of the smart glasses 214 and transmits it to a remote supporter via the specific processing unit 290 of the data processing device 12. The life management unit is realized by the control unit 46A of the smart glasses 214 and supports the user's daily life. The care assistance unit is realized by the control unit 46A of the smart glasses 214 and displays a visual guide for balancing using AR technology. The correspondence relationship between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0143] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0144] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0145] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0146] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0147] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0148] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0149] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0150] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0151] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0152] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0153] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0154] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0155] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0156] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0157] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0158] Each of the multiple elements including the vision adjustment unit, the hearing assistance unit, the view sharing unit, the life management unit, and the care assistance unit described above is realized, for example, by at least one of the headset type terminal 314 and the data processing device 12. For example, the vision adjustment unit is realized by the control unit 46A of the headset type terminal 314 and adjusts the user's vision in real time. The hearing assistance unit is realized by the control unit 46A of the headset type terminal 314 and amplifies surrounding sounds to assist the user's hearing. The view sharing unit photographs the user's field of view using the camera 42 of the headset type terminal 314 and transmits it to a remote supporter via the specific processing unit 290 of the data processing device 12. The life management unit is realized by the control unit 46A of the headset type terminal 314 and supports the user's daily life. The care assistance unit is realized by the control unit 46A of the headset type terminal 314 and displays a visual guide for balancing using AR technology. The correspondence relationship between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0159] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0160] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0161] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0162] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0163] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0164] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0165] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0166] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0167] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0168] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0169] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0170] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0171] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0172] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0173] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0174] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0175] Each of the multiple elements including the vision adjustment unit, the hearing assistance unit, the view sharing unit, the life management unit, and the care assistance unit described above is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the vision adjustment unit is realized by the control unit 46A of the robot 414 and adjusts the user's vision in real time. The hearing assistance unit is realized by the control unit 46A of the robot 414 and amplifies surrounding sounds to assist the user's hearing. The view sharing unit photographs the user's field of vision using the camera 42 of the robot 414 and transmits it to a remote supporter via the specific processing unit 290 of the data processing device 12. The life management unit is realized by the control unit 46A of the robot 414 and supports the user's daily life. The care assistance unit is realized by the control unit 46A of the robot 414 and displays a visual guide for balancing using AR technology. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0176] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0177] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0178] These emotions are distributed in the three o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0179] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0180] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0181] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0182] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0183] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0184] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0185] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0186] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0187] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0188] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0189] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0190] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0191] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0192] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0193] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0194] (Appendix 1) A vision adjustment unit that adjusts vision in real time; A hearing aid that amplifies surrounding sounds; A view sharing unit that communicates with a remote supporter in real time and shares the user's view; A life management section that supports the user's daily life; A care assistant unit that compensates for a decrease in balance or motor skills. A system characterized by: (Appendix 2) The vision adjustment unit is Using AR technology to enlarge visual information 2. The system of claim 1, (Appendix 3) The hearing assistance unit includes: Amplify surrounding sounds to hear them clearly 2. The system of claim 1, (Appendix 4) The view sharing unit includes: Communicate in real time with a remote supporter, who can give instructions while viewing the user's field of vision 2. The system of claim 1, (Appendix 5) The life management department includes: Reminds you when to take your medicine and displays your schedule 2. The system of claim 1, (Appendix 6) The care support department comprises: Uses AR technology to provide visual guidance for balancing 2. The system of claim 1, (Appendix 7) The vision adjustment unit is To estimate a user's emotion and adjust the frequency of vision adjustment based on the estimated user's emotion. 2. The system of claim 1, (Appendix 8) The vision adjustment unit is Analyze the user's past vision data and select the optimal vision adjustment method 2. The system of claim 1, (Appendix 9) The vision adjustment unit is During vision adjustment, the system adjusts how visual information is displayed based on the user's current activity 2. The system of claim 1, (Appendix 10) The vision adjustment unit is A user's emotion is estimated, and a priority of vision adjustment is determined based on the estimated user's emotion. 2. The system of claim 1, (Appendix 11) The vision adjustment unit is When adjusting vision, provide appropriate visual information by taking into account the user's geographic location information 2. The system of claim 1, (Appendix 12) The vision adjustment unit is Analyze users' social media activity and provide relevant visual information when adjusting vision 2. The system of claim 1, (Appendix 13) The hearing assistance unit includes: Estimating a user's emotion and adjusting the sound amplification level based on the estimated user's emotion 2. The system of claim 1, (Appendix 14) The hearing assistance unit includes: Analyze the user's past hearing data and select the optimal sound amplification method 2. The system of claim 1, (Appendix 15) The hearing assistance unit includes: During hearing assistance, sound filtering is performed based on the user's current environmental sounds. 2. The system of claim 1, (Appendix 16) The hearing assistance unit includes: A user's emotion is estimated, and a priority order for amplifying sounds is determined based on the estimated user's emotion. 2. The system of claim 1, (Appendix 17) The hearing assistance unit includes: When using hearing aids, sound amplification is performed taking into account the user's geographic location information. 2. The system of claim 1, (Appendix 18) The hearing assistance unit includes: When using hearing aids, the system analyzes the user's social media activity and provides relevant sound information 2. The system of claim 1, (Appendix 19) The view sharing unit includes: Estimate the user's emotions and adjust the frequency of view sharing based on the estimated user's emotions. 2. The system of claim 1, (Appendix 20) The view sharing unit includes: Analyze the user's past view sharing data and select the optimal view sharing method 2. The system of claim 1, (Appendix 21) The view sharing unit includes: When sharing visibility, adjust how visibility information is displayed based on the user's current activity 2. The system of claim 1, (Appendix 22) The view sharing unit includes: Estimate the user's emotions and determine the priority of view sharing based on the estimated user's emotions. 2. The system of claim 1, (Appendix 23) The view sharing unit includes: When sharing a view, provide view information taking into account the user's geographic location information 2. The system of claim 1, (Appendix 24) The view sharing unit includes: When sharing views, analyze users' social media activity and provide relevant view information 2. The system of claim 1, (Appendix 25) The life management department includes: Infer user emotions and adjust notification frequency based on the estimated user emotions 2. The system of claim 1, (Appendix 26) The life management department includes: Analyze the user's past lifestyle data and select the optimal notification method 2. The system of claim 1, (Appendix 27) The life management department includes: Tailor notifications based on the user's current activity when managing their life 2. The system of claim 1, (Appendix 28) The life management department includes: Infer user emotions and prioritize notifications based on the inferred user emotions 2. The system of claim 1, (Appendix 29) The life management department includes: When managing daily life, notifications are sent taking into account the user's geographic location information 2. The system of claim 1, (Appendix 30) The life management department includes: Analyze users' social media activity and provide relevant notifications when managing their life 2. The system of claim 1, (Appendix 31) The care support department comprises: To estimate a user's emotion and adjust the frequency of care assistance based on the estimated user's emotion. 2. The system of claim 1, (Appendix 32) The care support department comprises: Analyze the user's past care data and select the optimal care assistance method 2. The system of claim 1, (Appendix 33) The care support department comprises: During nursing care assistance, the assistance content is adjusted based on the user's current physical condition 2. The system of claim 1, (Appendix 34) The care support department comprises: Estimating the user's emotions and determining the priority of care assistance based on the estimated user's emotions 2. The system of claim 1, (Appendix 35) The care support department comprises: When providing nursing care assistance, the user's geographic location information is taken into consideration 2. The system of claim 1, (Appendix 36) The care support department comprises: Analyzing users' social media activities and providing relevant care information during care support 2. The system of claim 1, [Explanation of symbols]

[0195] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A vision adjustment unit that adjusts vision in real time; A hearing aid that amplifies surrounding sounds; A view sharing unit that communicates with a remote supporter in real time and shares the user's view; A life management section that supports the user's daily life; A care assistant unit that compensates for a decrease in balance or motor skills. A system characterized by:

2. The vision adjustment unit is Using AR technology to enlarge visual information 2. The system of claim 1 .

3. The hearing assistance unit includes: Amplify surrounding sounds to hear them clearly 2. The system of claim 1 .

4. The view sharing unit includes: Communicate in real time with a remote supporter, who can give instructions while viewing the user's field of vision 2. The system of claim 1 .

5. The life management department includes: Reminds you when to take your medicine and displays your schedule 2. The system of claim 1 .

6. The care support department comprises: Uses AR technology to provide visual guidance for balancing 2. The system of claim 1 .

7. The vision adjustment unit is To estimate a user's emotion and adjust the frequency of vision adjustment based on the estimated user's emotion.

2. The system of claim 1 .

8. The vision adjustment unit is Analyze the user's past vision data and select the optimal vision adjustment method 2. The system of claim 1 .

Images