system

Abstract

The system according to this embodiment aims to comprehensively understand the mental health status of the subject and guide them to appropriate counseling. [Solution] The system according to the embodiment comprises a face recognition unit, a voice recognition unit, a wearable device unit, a camera analysis unit, an integrated analysis unit, and a counseling guidance unit. The face recognition unit analyzes the subject's complexion and facial expressions. The voice recognition unit analyzes the subject's voice tone and volume. The wearable device unit collects the subject's biometric information. The camera analysis unit analyzes the subject's behavior. The integrated analysis unit comprehensively analyzes the data collected by the face recognition unit, voice recognition unit, wearable device unit, and camera analysis unit. The counseling guidance unit guides the subject to counseling based on the results analyzed by the integrated analysis unit.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, it is difficult to comprehensively grasp the mental health state of the target person, and there is room for improvement.

[0005] The system according to the embodiment aims to comprehensively grasp the mental health state of the target person and guide appropriate counseling.

Means for Solving the Problems

[0006] The system according to this embodiment comprises a facial recognition unit, a voice recognition unit, a wearable device unit, a camera analysis unit, an integrated analysis unit, and a counseling guidance unit. The facial recognition unit analyzes the subject's complexion and facial expressions. The voice recognition unit analyzes the subject's voice tone and volume. The wearable device unit collects the subject's biometric information. The camera analysis unit analyzes the subject's behavior. The integrated analysis unit comprehensively analyzes the data collected by the facial recognition unit, voice recognition unit, wearable device unit, and camera analysis unit. The counseling guidance unit guides the subject to counseling based on the results analyzed by the integrated analysis unit. [Effects of the Invention]

[0007] The system according to this embodiment can comprehensively assess the mental health status of the subject and guide them to appropriate counseling. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 comprises a computer 36, a receiving device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The receiving device 38, output device 40, and camera 42 are also connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The mental health status assessment system according to an embodiment of the present invention is a system that assesses the mental health status of a subject from the characteristics of their voice, complexion, and behavior, using facial recognition, voice recognition, wearable devices, camera solutions, etc. The mental health status assessment system can detect depression early, assess the state of mental health, and notice changes in the state that the person is unaware of. First, facial recognition technology is used to analyze the subject's complexion and facial expressions. For example, changes in complexion and subtle changes in facial expressions are detected, and the mental health status is estimated based on this data. Next, voice recognition technology is used to analyze the tone and volume of the subject's voice. For example, changes in voice tone or volume can be detected to indicate a change in mental health status. Furthermore, wearable devices are used to collect biometric information from the subject. For example, data such as heart rate, body temperature, and activity level are collected, and the mental health status is estimated based on this data. It is also possible to analyze the subject's behavior using a camera solution. For example, changes in walking style and posture are detected, and the mental health status is estimated based on this data. This data is comprehensively analyzed by AI, allowing for a complete understanding of the subject's mental health status. For example, if changes in facial color, voice tone, and heart rate are detected simultaneously, it is determined that there is a high probability of depression. In this way, the AI ​​can monitor the subject's mental health in real time and take appropriate action, such as guiding them to counseling, as needed. This system allows subjects to accurately understand their own mental health and take appropriate action. For example, early detection of depression allows for early initiation of treatment. Also, by noticing changes in mental health, they can re-evaluate their stress management and relaxation methods. This makes it possible to live in a state of constant mental and physical health. In this way, the mental health monitoring system can comprehensively understand the subject's mental health and take appropriate action.

[0029] The mental health status assessment system according to this embodiment comprises a facial recognition unit, a voice recognition unit, a wearable device unit, a camera analysis unit, an integrated analysis unit, and a counseling guidance unit. The facial recognition unit analyzes the subject's complexion and facial expressions. The facial recognition unit detects, for example, changes in complexion and subtle changes in facial expressions. To detect changes in complexion, the facial recognition unit can analyze changes in hue and brightness. To detect subtle changes in facial expressions, the facial recognition unit can analyze eyebrow movements and the raising and lowering of the corners of the mouth. The voice recognition unit analyzes the subject's voice tone and volume. The voice recognition unit detects, for example, changes in voice tone and volume. To detect changes in voice tone, the voice recognition unit can analyze changes in pitch and intonation. To detect changes in voice volume, the voice recognition unit can analyze changes in decibels and sound pressure. The wearable device unit collects the subject's biometric information. The wearable device unit collects data such as heart rate, body temperature, and activity level. The wearable device unit can analyze the range and variability patterns of heart rate in order to collect heart rate data. The wearable device unit can analyze the range and variability patterns of body temperature in order to collect body temperature data. The wearable device unit can analyze steps taken and exercise intensity in order to collect activity levels. The camera analysis unit analyzes the behavior of the subject. For example, the camera analysis unit detects changes in walking style and posture. For detecting walking style, the camera analysis unit can analyze stride length and walking speed. For detecting changes in posture, the camera analysis unit can analyze the degree of spinal straightness and shoulder position. The integrated analysis unit comprehensively analyzes the data collected by the facial recognition unit, voice recognition unit, wearable device unit, and camera analysis unit. For example, the integrated analysis unit comprehensively analyzes data such as changes in facial color, changes in voice tone, and changes in heart rate to estimate mental health status. The integrated analysis unit can estimate mental health status using data weighting and analysis algorithms. The counseling guidance unit guides the user to counseling based on the results analyzed by the integrated analysis unit. For example, the counseling guidance unit guides the user to counseling if their mental health is deteriorating.The counseling guidance unit can adjust the timing, method, and content of the guidance. As a result, the mental health status assessment system according to this embodiment can comprehensively understand the mental health status of the subject and take appropriate action.

[0030] The facial recognition unit analyzes the subject's complexion and facial expressions. Specifically, the facial recognition unit uses a high-resolution camera to capture the subject's face and analyzes the video data in real time. To detect changes in complexion, it can analyze changes in hue and brightness. For example, increased redness in the complexion may indicate stress or excitement, while a pale complexion may indicate fatigue or anxiety. To detect these changes, the facial recognition unit analyzes fluctuations in RGB values ​​and uses an algorithm to identify specific patterns. Furthermore, to detect subtle changes in facial expression, it can analyze eyebrow movements and the ups and downs of the corners of the mouth. For example, furrowed brows may indicate anxiety or tension, while drooping corners of the mouth may indicate sadness or depression. To detect these changes in facial expression, the facial recognition unit tracks the movement of each part of the face and analyzes subtle changes with high accuracy. In addition, the facial recognition unit can accumulate this data and monitor long-term changes. This makes it possible to grasp trends in the subject's mental health and detect abnormalities early.

[0031] The speech recognition unit analyzes the tone and volume of the subject's voice. Specifically, the speech recognition unit uses a high-sensitivity microphone to collect the subject's voice and analyzes the audio data in real time. To detect changes in voice tone, it can analyze changes in pitch and intonation. For example, a higher voice tone may indicate excitement or tension, while a lower tone may indicate calmness or fatigue. To detect these changes, the speech recognition unit analyzes the frequency components of the voice waveform and uses an algorithm to identify specific patterns. Furthermore, to detect changes in voice volume, it can analyze changes in decibels and sound pressure. For example, a rapid change in voice volume may indicate emotional fluctuations, while maintaining a constant volume may indicate a stable state. To detect these changes, the speech recognition unit analyzes the amplitude of the voice and uses an algorithm to identify specific patterns. In addition, the speech recognition unit can accumulate this data and monitor long-term changes. This makes it possible to grasp trends in the subject's mental health and detect abnormalities early.

[0032] The wearable device unit collects biometric information from the subject. Specifically, the wearable device unit uses a device worn by the subject to collect data such as heart rate, body temperature, and activity level. To collect heart rate, a heart rate sensor can be used to analyze the range and fluctuation patterns of the heart rate. For example, a rapid increase in heart rate may indicate stress or excitement, while a stable heart rate within a certain range may indicate relaxation. To detect these changes, the wearable device unit analyzes time-series data of heart rate and uses an algorithm to identify specific patterns. Similarly, to collect body temperature, a body temperature sensor can be used to analyze the range and fluctuation patterns of body temperature. For example, an increase in body temperature may indicate fever or stress, while a stable body temperature within a certain range may indicate a healthy state. To detect these changes, the wearable device unit analyzes time-series data of body temperature and uses an algorithm to identify specific patterns. Furthermore, to collect activity level, an accelerometer and gyroscope can be used to analyze steps taken and exercise intensity. For example, an increase in activity level may indicate an active state, while a decrease may indicate fatigue or depression. To detect these changes, the wearable device analyzes time-series data of activity levels and uses an algorithm to identify specific patterns. This allows the wearable device to collect biometric information of the subject with high accuracy and comprehensively understand their mental health status.

[0033] The camera analysis unit analyzes the behavior of the subject. Specifically, the camera analysis unit uses a high-resolution camera to capture the subject's movements and analyzes the video data in real time. To detect walking patterns, it can analyze stride length and walking speed. For example, a shorter stride and slower walking speed may indicate fatigue or depression, while a longer stride and faster walking speed may indicate an active state. To detect these changes, the camera analysis unit tracks the subject's movements from the video data and uses an algorithm to identify specific patterns. Furthermore, to detect changes in posture, it can analyze the degree of spinal straightness and shoulder position. For example, a curved spine and drooping shoulders may indicate fatigue or depression, while a straight spine and raised shoulders may indicate confidence and vitality. To detect these changes, the camera analysis unit analyzes the subject's posture from the video data and uses an algorithm to identify specific patterns. In addition, the camera analysis unit can accumulate this data and monitor long-term changes. This makes it possible to grasp trends in the subject's mental health and detect abnormalities early.

[0034] The integrated analysis unit comprehensively analyzes data collected by the facial recognition unit, voice recognition unit, wearable device unit, and camera analysis unit. Specifically, the integrated analysis unit centrally manages the data collected from each unit and estimates mental health status using data weighting and analysis algorithms. For example, it comprehensively analyzes data such as changes in facial color, changes in voice tone, and changes in heart rate to estimate mental health status. The integrated analysis unit analyzes this data over time, allowing it to grasp not only short-term changes but also long-term trends. Furthermore, the integrated analysis unit can use anomaly detection algorithms to detect unusual patterns and abnormal data, and issue warnings early. For example, if a sudden increase in heart rate or a sudden change in voice tone is detected, the integrated analysis unit immediately detects the anomaly and prompts appropriate action. The integrated analysis unit can also utilize historical data and statistical information to perform long-term risk assessment and trend analysis. As a result, the integrated analysis unit can not only grasp the situation in real time but also handle long-term risk management and anomaly detection, improving the reliability and safety of the entire system.

[0035] The Counseling Guidance Unit guides individuals to counseling based on the results analyzed by the Integrated Analysis Unit. Specifically, the Counseling Guidance Unit guides individuals to counseling when their mental health is deteriorating. For example, if an individual's mental health falls below a certain standard, the Counseling Guidance Unit sends a notification to the individual prompting them to schedule a counseling session. This notification is sent via a smartphone app, email, or SMS. The Counseling Guidance Unit can also adjust the timing, method, and content of the guidance. For example, it considers the individual's schedule and past counseling history to send a notification at the optimal time. The content of the notification is also customized according to the individual's condition. For example, it suggests relaxation methods for mild stressors and recommends a consultation with a professional counselor for severe stressors. Furthermore, the Counseling Guidance Unit collects feedback from individuals and can continuously improve the accuracy and effectiveness of the guidance. This allows the Counseling Guidance Unit to quickly provide appropriate counseling guidance to individuals and support the improvement of their mental health.

[0036] The facial recognition unit can detect changes in skin tone and subtle changes in facial expression. For example, to detect changes in skin tone, the facial recognition unit analyzes changes in hue and brightness. To detect subtle changes in facial expression, the facial recognition unit can analyze eyebrow movements and the ups and downs of the corners of the mouth. By analyzing changes in skin tone and facial expression in detail, it is possible to estimate mental health more accurately. Some or all of the above processing in the facial recognition unit may be performed using AI, for example, or without AI. For example, the facial recognition unit can perform analysis using generative AI to detect changes in skin tone and subtle changes in facial expression.

[0037] The speech recognition unit can detect changes in voice tone and volume. For example, to detect changes in voice tone, the speech recognition unit can analyze changes in pitch and intonation. To detect changes in voice volume, the speech recognition unit can analyze changes in decibels and sound pressure. By analyzing changes in voice tone and volume in detail, it is possible to estimate mental health status more accurately. Some or all of the above processing in the speech recognition unit may be performed using AI, for example, or without AI. For example, the speech recognition unit can perform analysis using generative AI to detect changes in voice tone and volume.

[0038] The wearable device can collect biometric information such as heart rate, body temperature, and activity level. For example, to collect heart rate, the wearable device can analyze the range and fluctuation patterns of heart rate. To collect body temperature, the wearable device can analyze the range and fluctuation patterns of body temperature. To collect activity level, the wearable device can analyze steps taken and exercise intensity. By collecting biometric information in this way, the mental health state can be estimated more accurately. Some or all of the above processing in the wearable device may be performed using AI, for example, or without AI. For example, to collect biometric information such as heart rate, body temperature, and activity level, the wearable device can perform analysis using generative AI.

[0039] The camera analysis unit can detect changes in walking style and posture. For example, to detect walking style, the camera analysis unit can analyze stride length and walking speed. To detect changes in posture, the camera analysis unit can analyze the degree of spinal straightness and shoulder position. By analyzing changes in behavior in detail, it is possible to estimate mental health status more accurately. Some or all of the above processing in the camera analysis unit may be performed using AI, for example, or without AI. For example, the camera analysis unit can perform analysis using generative AI to detect changes in walking style and posture.

[0040] The integrated analysis unit can comprehensively analyze data such as changes in facial color, voice tone, and heart rate to estimate mental health status. The integrated analysis unit estimates mental health status using, for example, data weighting and analysis algorithms. This allows for a more accurate estimation of mental health status by comprehensively analyzing multiple data. Some or all of the above-described processes in the integrated analysis unit may be performed using, for example, AI, or not. For example, the integrated analysis unit can use generative AI to perform analysis in order to comprehensively analyze data such as changes in facial color, voice tone, and heart rate.

[0041] The counseling guidance unit can guide the user to counseling as needed, based on the results analyzed by the integrated analysis unit. For example, the counseling guidance unit will guide the user to counseling if their mental health is deteriorating. The counseling guidance unit can adjust the timing, method, and content of the guidance. This allows for improvement of the user's mental health by providing appropriate guidance to counseling based on the analysis results. Some or all of the above-described processes in the counseling guidance unit may be performed using AI, for example, or without AI. For example, the counseling guidance unit can input the results analyzed by the integrated analysis unit into a generating AI and have the generating AI execute a method for guiding the user to counseling.

[0042] The facial recognition unit can estimate the subject's emotions and analyze changes in facial color in more detail based on the estimated emotions. For example, if the subject is stressed, the facial recognition unit can analyze changes in redness or blueness of the facial color in detail. If the subject is relaxed, the facial recognition unit can analyze the uniformity and healthiness of the facial color in detail. If the subject is tense, the facial recognition unit can analyze subtle changes in facial color and blood flow in detail. This allows for a more accurate estimation of mental health by analyzing changes in facial color in detail based on emotions. Some or all of the above processing in the facial recognition unit may be performed using AI, for example, or without AI. For example, the facial recognition unit can use generative AI to estimate the subject's emotions and analyze changes in facial color in detail based on the estimated emotions.

[0043] The facial recognition unit can analyze the moisture content and blood flow of the subject's skin during facial recognition and estimate their mental health. For example, the facial recognition unit can analyze the moisture content of the subject's skin and evaluate its dryness or degree of moisture. The facial recognition unit can analyze the subject's blood flow and evaluate its state of poor or improved blood circulation. The facial recognition unit can combine changes in the subject's skin moisture content and blood flow to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing skin moisture content and blood flow. Some or all of the above processing in the facial recognition unit may be performed using AI, for example, or without AI. For example, the facial recognition unit can use generative AI to analyze the moisture content and blood flow of the subject's skin during facial recognition.

[0044] The facial recognition unit can analyze the subject's eye movements and blinking frequency during facial recognition to estimate their mental health. For example, the facial recognition unit can analyze the subject's eye movements and evaluate the movement of their gaze and how well they focus. The facial recognition unit can analyze the subject's blinking frequency to evaluate the degree of tension and fatigue. By combining the subject's eye movements and blinking frequency, the facial recognition unit can comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing eye movements and blinking frequency. Some or all of the above-described processes in the facial recognition unit may be performed using AI, for example, or without AI. For example, the facial recognition unit can use generative AI to analyze the subject's eye movements and blinking frequency during facial recognition.

[0045] The facial recognition unit can estimate the subject's emotions and analyze changes in facial expression in real time based on the estimated emotions. For example, if the subject is happy, the facial recognition unit can analyze the degree of their smile and the brightness of their eyes in real time. If the subject is sad, the facial recognition unit can analyze the degree to which the corners of their mouth are turned down and the moisture in their eyes in real time. If the subject is surprised, the facial recognition unit can analyze the degree to which their eyes are open and the movement of their eyebrows in real time. This allows for a more accurate estimation of mental health by analyzing changes in facial expression in real time based on emotions. Some or all of the above processing in the facial recognition unit may be performed using AI, for example, or without AI. For example, the facial recognition unit can use generative AI to estimate the subject's emotions and analyze changes in facial expression in real time based on the estimated emotions.

[0046] The facial recognition unit can analyze the temperature distribution of a subject's face during facial recognition and estimate their mental health. For example, the facial recognition unit can analyze the temperature distribution of a subject's face and evaluate areas that tend to get cold easily and areas that tend to get warm easily. The facial recognition unit can analyze changes in the temperature distribution of a subject's face and evaluate their stress and relaxation levels. The facial recognition unit can combine the temperature distribution of a subject's face with other biometric information to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing the temperature distribution of the face. Some or all of the above-described processes in the facial recognition unit may be performed using AI, for example, or without AI. For example, the facial recognition unit can use generative AI to analyze the temperature distribution of a subject's face during facial recognition.

[0047] The facial recognition unit can analyze the tension of the subject's facial muscles during facial recognition and estimate their mental health. For example, the facial recognition unit can analyze the tension of the subject's facial muscles and evaluate whether they are relaxed or tense. The facial recognition unit can analyze changes in the tension of the subject's facial muscles and evaluate the degree of stress or fatigue. The facial recognition unit can combine the tension of the subject's facial muscles with other biometric information to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing the tension of facial muscles. Some or all of the above-described processes in the facial recognition unit may be performed using AI, for example, or without AI. For example, the facial recognition unit can use generative AI to analyze the tension of the subject's facial muscles during facial recognition.

[0048] The speech recognition unit can estimate the subject's emotions and analyze changes in voice tone in more detail based on the estimated emotions. For example, if the subject is nervous, the speech recognition unit can analyze the pitch and tremor of the voice tone in detail. If the subject is relaxed, the speech recognition unit can analyze the stability and smoothness of the voice tone in detail. If the subject is excited, the speech recognition unit can analyze the rapid changes and intensity of the voice tone in detail. This allows for a more accurate estimation of mental health by analyzing changes in voice tone in detail based on emotions. Some or all of the above processing in the speech recognition unit may be performed using AI, for example, or without AI. For example, the speech recognition unit can use generative AI to estimate the subject's emotions and analyze changes in voice tone in detail based on the estimated emotions.

[0049] The speech recognition unit can analyze the subject's breathing sounds and cough frequency during speech recognition to estimate their mental health. For example, the speech recognition unit can analyze the subject's breathing sounds and evaluate the depth and rhythm of their breathing. The speech recognition unit can analyze the subject's cough frequency and evaluate the degree of stress and fatigue. The speech recognition unit can combine the subject's breathing sounds and cough frequency to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing breathing sounds and cough frequency. Some or all of the above processing in the speech recognition unit may be performed using AI, for example, or without AI. For example, the speech recognition unit can use generative AI to analyze the subject's breathing sounds and cough frequency during speech recognition.

[0050] The speech recognition unit can analyze the speed and rhythm of a person's speech during speech recognition and estimate their mental health. For example, the speech recognition unit can analyze the speed of the person's speech and evaluate the degree of tension or relaxation. The speech recognition unit can analyze the rhythm of the person's speech and evaluate the degree of stress or fatigue. The speech recognition unit can combine the speed and rhythm of the person's speech to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing the speed and rhythm of speech. Some or all of the above processing in the speech recognition unit may be performed using AI, for example, or without AI. For example, the speech recognition unit can use generative AI to analyze the speed and rhythm of the person's speech during speech recognition.

[0051] The speech recognition unit can estimate the subject's emotions and analyze changes in voice volume in real time based on the estimated emotions. For example, if the subject is nervous, the speech recognition unit can analyze changes in voice volume in real time. If the subject is relaxed, the speech recognition unit can analyze the stability of voice volume in real time. If the subject is excited, the speech recognition unit can analyze rapid changes in voice volume in real time. This allows for a more accurate estimation of mental health by analyzing changes in voice volume in real time based on emotions. Some or all of the above processing in the speech recognition unit may be performed using AI, for example, or without AI. For example, the speech recognition unit can use generative AI to estimate the subject's emotions and analyze changes in voice volume in real time based on the estimated emotions.

[0052] The speech recognition unit can analyze the frequency components of a subject's voice during speech recognition and estimate their mental health. For example, the speech recognition unit can analyze the frequency components of a subject's voice and evaluate the degree of tension or relaxation. The speech recognition unit can analyze changes in the frequency components of a subject's voice and evaluate the degree of stress or fatigue. The speech recognition unit can combine the frequency components of a subject's voice with other biometric information to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing the frequency components of the voice. Some or all of the above-described processes in the speech recognition unit may be performed using AI, for example, or without AI. For example, the speech recognition unit can use generative AI to analyze the frequency components of a subject's voice during speech recognition.

[0053] The speech recognition unit can analyze the intonation and pitch of a subject's voice during speech recognition to estimate their mental health. For example, the speech recognition unit can analyze the intonation of a subject's voice to evaluate the degree of tension or relaxation. The speech recognition unit can analyze the pitch of a subject's voice to evaluate the degree of stress or fatigue. The speech recognition unit can combine the intonation and pitch of a subject's voice to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing the intonation and pitch of the voice. Some or all of the above processing in the speech recognition unit may be performed using AI, for example, or without AI. For example, the speech recognition unit can use generative AI to analyze the intonation and pitch of a subject's voice during speech recognition.

[0054] The wearable device can estimate the subject's emotions and analyze changes in heart rate in more detail based on the estimated emotions. For example, if the subject is tense, the wearable device can analyze rapid changes in heart rate in detail. If the subject is relaxed, the wearable device can analyze the stability of heart rate in detail. If the subject is excited, the wearable device can analyze rapid increases in heart rate in detail. This allows for a more accurate estimation of mental health by analyzing changes in heart rate in detail based on emotions. Some or all of the above processing in the wearable device may be performed using AI, for example, or without AI. For example, the wearable device can use generative AI to estimate the subject's emotions and analyze changes in heart rate in detail based on the estimated emotions.

[0055] The wearable device unit can analyze a subject's blood pressure and blood glucose levels to estimate their mental health. For example, the wearable device unit can analyze a subject's blood pressure to assess their stress and relaxation levels. The wearable device unit can analyze a subject's blood glucose levels to assess their fatigue and energy levels. The wearable device unit can combine the subject's blood pressure and blood glucose levels to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing blood pressure and blood glucose levels. Some or all of the above-described processes in the wearable device unit may be performed using AI, for example, or without AI. For example, the wearable device unit can use generative AI to analyze a subject's blood pressure and blood glucose levels.

[0056] The wearable device unit can analyze a subject's sleep patterns and estimate their mental health. For example, the wearable device unit can analyze a subject's sleep patterns and evaluate the quality and quantity of sleep. The wearable device unit can analyze changes in a subject's sleep patterns and evaluate the degree of stress and fatigue. The wearable device unit can combine a subject's sleep patterns with other biometric information to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing sleep patterns. Some or all of the above-described processes in the wearable device unit may be performed using AI, for example, or without AI. For example, the wearable device unit can use generative AI to analyze a subject's sleep patterns.

[0057] The wearable device can estimate the subject's emotions and analyze changes in body temperature in real time based on the estimated emotions. For example, if the subject is tense, the wearable device can analyze rapid changes in body temperature in real time. If the subject is relaxed, the wearable device can analyze the stability of body temperature in real time. If the subject is excited, the wearable device can analyze rapid increases in body temperature in real time. This allows for a more accurate estimation of mental health by analyzing changes in body temperature in real time based on emotions. Some or all of the above processing in the wearable device may be performed using AI, for example, or without AI. For example, the wearable device can use generative AI to estimate the subject's emotions and analyze changes in body temperature in real time based on the estimated emotions.

[0058] The wearable device unit can analyze the subject's skin electrical activity and estimate their mental health state. For example, the wearable device unit can analyze the subject's skin electrical activity and evaluate the degree of stress and relaxation. The wearable device unit can analyze changes in the subject's skin electrical activity and evaluate the degree of tension and fatigue. The wearable device unit can combine the subject's skin electrical activity with other biometric information to comprehensively estimate their mental health state. This allows for a more accurate estimation of mental health state by analyzing skin electrical activity. Some or all of the above-described processes in the wearable device unit may be performed using AI, for example, or without AI. For example, the wearable device unit can use generative AI to analyze the subject's skin electrical activity.

[0059] The wearable device unit can analyze changes in a subject's activity level and estimate their mental health. For example, the wearable device unit can analyze a subject's activity level and evaluate their daily exercise and activity patterns. The wearable device unit can analyze changes in a subject's activity level and evaluate their stress and fatigue levels. The wearable device unit can combine a subject's activity level with other biometric information to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing changes in activity level. Some or all of the above-described processes in the wearable device unit may be performed using AI, for example, or without AI. For example, the wearable device unit can use generative AI to analyze changes in a subject's activity level.

[0060] The camera analysis unit can estimate the subject's emotions and analyze changes in their gait in more detail based on those estimated emotions. For example, if the subject is tense, the camera analysis unit can analyze changes in walking speed and rhythm in detail. If the subject is relaxed, the camera analysis unit can analyze the stability and smoothness of their gait in detail. If the subject is excited, the camera analysis unit can analyze sudden changes and variations in gait in detail. This allows for a more accurate estimation of mental health by analyzing changes in gait based on emotions. Some or all of the above processing in the camera analysis unit may be performed using AI, for example, or without AI. For example, the camera analysis unit can use generative AI to estimate the subject's emotions and analyze changes in their gait in detail based on those estimated emotions.

[0061] The camera analysis unit can analyze subtle changes in a subject's posture during camera analysis and estimate their mental health. For example, the camera analysis unit can analyze subtle changes in a subject's posture and evaluate their level of tension or relaxation. The camera analysis unit can analyze changes in a subject's posture and evaluate their level of stress or fatigue. The camera analysis unit can combine subtle changes in a subject's posture with other biometric information to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing subtle changes in posture. Some or all of the above-described processes in the camera analysis unit may be performed using AI, for example, or without AI. For example, the camera analysis unit can use generative AI to analyze subtle changes in a subject's posture during camera analysis.

[0062] The camera analysis unit can analyze the subject's hand movements and gestures during camera analysis to estimate their mental health. For example, the camera analysis unit can analyze the subject's hand movements to evaluate their level of tension or relaxation. The camera analysis unit can analyze the subject's gestures to evaluate their level of stress or fatigue. The camera analysis unit can combine the subject's hand movements and gestures to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing hand movements and gestures. Some or all of the above processing in the camera analysis unit may be performed using AI, for example, or without AI. For example, the camera analysis unit can use generative AI to analyze the subject's hand movements and gestures during camera analysis.

[0063] The camera analysis unit can estimate the subject's emotions and analyze changes in posture in real time based on the estimated emotions. For example, if the subject is tense, the camera analysis unit can analyze changes in posture in real time. If the subject is relaxed, the camera analysis unit can analyze the stability of their posture in real time. If the subject is excited, the camera analysis unit can analyze rapid changes in posture in real time. This allows for a more accurate estimation of mental health by analyzing changes in posture in real time based on emotions. Some or all of the above processing in the camera analysis unit may be performed using AI, for example, or without AI. For example, the camera analysis unit can use generative AI to estimate the subject's emotions and analyze changes in posture in real time based on the estimated emotions.

[0064] The camera analysis unit can analyze the speed and rhythm of a subject's movements during camera analysis and estimate their mental health. For example, the camera analysis unit can analyze the speed of a subject's movements and evaluate their level of tension or relaxation. The camera analysis unit can analyze the rhythm of a subject's movements and evaluate their level of stress or fatigue. The camera analysis unit can combine the speed and rhythm of a subject's movements to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing the speed and rhythm of movements. Some or all of the above-described processes in the camera analysis unit may be performed using AI, for example, or without AI. For example, the camera analysis unit can use generative AI to analyze the speed and rhythm of a subject's movements during camera analysis.

[0065] The camera analysis unit can analyze the consistency and coordination of a subject's movements during camera analysis and estimate their mental health. For example, the camera analysis unit can analyze the consistency of a subject's movements and evaluate their level of tension and relaxation. The camera analysis unit can analyze the coordination of a subject's movements and evaluate their level of stress and fatigue. By combining the consistency and coordination of a subject's movements, the camera analysis unit can comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing the consistency and coordination of movements. Some or all of the above-described processes in the camera analysis unit may be performed using AI, for example, or without AI. For example, the camera analysis unit can use generative AI to analyze the consistency and coordination of a subject's movements during camera analysis.

[0066] The integrated analysis unit can estimate the subject's emotions and adjust the integrated analysis algorithm based on the estimated emotions. For example, if the subject is tense, the integrated analysis unit will prioritize data related to tension in its analysis. If the subject is relaxed, the integrated analysis unit can prioritize data related to relaxation in its analysis. If the subject is excited, the integrated analysis unit can prioritize data related to excitement in its analysis. By adjusting the integrated analysis algorithm based on emotions, the mental health state can be estimated more accurately. Some or all of the above processing in the integrated analysis unit may be performed using AI, for example, or without AI. For example, the integrated analysis unit can use generative AI to perform analysis in order to estimate the subject's emotions and adjust the integrated analysis algorithm based on the estimated emotions.

[0067] The integrated analysis unit can predict the current mental health state by referring to past data during integrated analysis. For example, the integrated analysis unit can predict the current mental health state by referring to past data. The integrated analysis unit can predict changes in the mental health state by comparing past data with current data. The integrated analysis unit can predict the future mental health state based on past data. This allows for a more accurate prediction of the current mental health state by referring to past data. Some or all of the above processes in the integrated analysis unit may be performed using AI, for example, or without AI. For example, the integrated analysis unit can use generative AI to perform analysis in order to predict the current mental health state by referring to past data during integrated analysis.

[0068] The integrated analysis unit can analyze correlations between different data sources during integrated analysis and estimate mental health status. For example, the integrated analysis unit can analyze the correlation between changes in facial color and changes in voice tone to estimate mental health status. The integrated analysis unit can analyze the correlation between changes in heart rate and changes in gait to estimate mental health status. The integrated analysis unit can analyze the correlation between changes in skin electrical activity and changes in posture to estimate mental health status. In this way, mental health status can be estimated more accurately by analyzing correlations between different data sources. Some or all of the above processing in the integrated analysis unit may be performed using AI, for example, or without AI. For example, the integrated analysis unit can use generative AI to analyze correlations between different data sources during integrated analysis.

[0069] The integrated analysis unit can estimate the subject's emotions and adjust the order in which the results of the integrated analysis are displayed based on the estimated emotions. For example, if the subject is tense, the integrated analysis unit can prioritize displaying data related to tension. If the subject is relaxed, the integrated analysis unit can prioritize displaying data related to relaxation. If the subject is excited, the integrated analysis unit can prioritize displaying data related to excitement. By adjusting the order in which results are displayed based on emotions, the mental health state can be estimated more accurately. Some or all of the above processing in the integrated analysis unit may be performed using AI, for example, or without AI. For example, the integrated analysis unit can use generative AI to perform analysis in order to estimate the subject's emotions and adjust the order in which the results of the integrated analysis are displayed based on the estimated emotions.

[0070] The integrated analysis unit can estimate the mental health status of a subject by referring to their living environment data during integrated analysis. For example, the integrated analysis unit can estimate the mental health status by referring to the subject's living environment data. The integrated analysis unit can comprehensively estimate the mental health status by combining the subject's living environment data with other biometric information. The integrated analysis unit can predict the future mental health status based on the subject's living environment data. This allows for a more accurate estimation of mental health status by referring to living environment data. Some or all of the above-described processes in the integrated analysis unit may be performed using AI, for example, or without AI. For example, the integrated analysis unit can use generative AI to perform analysis in order to estimate the mental health status by referring to the subject's living environment data during integrated analysis.

[0071] The integrated analysis unit can estimate the mental health status of a subject by referring to their social relationship data during integrated analysis. For example, the integrated analysis unit can estimate the mental health status by referring to the subject's social relationship data. The integrated analysis unit can comprehensively estimate the mental health status by combining the subject's social relationship data with other biometric information. The integrated analysis unit can predict the future mental health status of a subject based on their social relationship data. This allows for a more accurate estimation of mental health status by referring to social relationship data. Some or all of the above-described processes in the integrated analysis unit may be performed using AI, for example, or without AI. For example, the integrated analysis unit can use generative AI to perform analysis in order to estimate the mental health status of a subject by referring to their social relationship data during integrated analysis.

[0072] The counseling guidance unit can estimate the client's emotions and adjust the counseling guidance method based on the estimated emotions. For example, if the client is tense, the counseling guidance unit can provide a guidance method that helps them relax. If the client is relaxed, the counseling guidance unit can provide detailed counseling. If the client is agitated, the counseling guidance unit can provide a guidance method that helps them calm down. In this way, by adjusting the counseling guidance method based on emotions, more appropriate counseling can be provided. Some or all of the above processing in the counseling guidance unit may be performed using AI, for example, or without AI. For example, the counseling guidance unit can use generative AI to perform analysis in order to estimate the client's emotions and adjust the counseling guidance method based on the estimated emotions.

[0073] The counseling guidance unit can select the optimal guidance method by referring to the subject's past counseling history during counseling guidance. For example, the counseling guidance unit can refer to the subject's past counseling history and select the optimal guidance method. The counseling guidance unit can combine the subject's past counseling history with current data to select the optimal guidance method. The counseling guidance unit can predict future counseling methods based on the subject's past counseling history. This allows for the provision of more appropriate counseling by referring to past counseling history. Some or all of the above processing in the counseling guidance unit may be performed using AI, for example, or without AI. For example, the counseling guidance unit can use generative AI to perform analysis in order to select the optimal guidance method by referring to the subject's past counseling history during counseling guidance.

[0074] The counseling guidance unit can estimate the client's emotions and determine the priority of counseling based on the estimated emotions. For example, if the client is tense, the counseling guidance unit may prioritize counseling that helps them relax. If the client is relaxed, the counseling guidance unit may prioritize detailed counseling. If the client is agitated, the counseling guidance unit may prioritize counseling that helps them calm down. By determining the priority of counseling based on emotions, more appropriate counseling can be provided. Some or all of the above processing in the counseling guidance unit may be performed using AI, for example, or without AI. For example, the counseling guidance unit may use generative AI to perform analysis in order to estimate the client's emotions and determine the priority of counseling based on the estimated emotions.

[0075] The counseling guidance unit can select the optimal guidance method by referring to the subject's living environment data during counseling guidance. For example, the counseling guidance unit can select the optimal guidance method by referring to the subject's living environment data. The counseling guidance unit can select the optimal guidance method by combining the subject's living environment data with other biometric information. The counseling guidance unit can predict future guidance methods based on the subject's living environment data. This allows for more appropriate counseling to be provided by referring to the living environment data. Some or all of the above processing in the counseling guidance unit may be performed using AI, for example, or without AI. For example, the counseling guidance unit can use generative AI to perform analysis in order to select the optimal guidance method by referring to the subject's living environment data during counseling guidance.

[0076] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0077] The mental health assessment system can further collect dietary data from subjects and estimate their mental health status. For example, it can analyze the content of meals, calorie intake, and nutritional balance to identify factors that influence mental health. By combining dietary data with other biometric information, it can more accurately estimate mental health status. For instance, it can evaluate the impact of nutritional deficiencies or excesses on mental health and provide appropriate dietary guidance. It can also analyze the timing and frequency of meals to find patterns related to mental health. This allows for a comprehensive understanding of mental health status using dietary data, enabling appropriate responses.

[0078] The mental health assessment system can further collect exercise data from subjects and estimate their mental health status. For example, it can analyze the type, frequency, and intensity of exercise to identify factors that influence mental health. By combining exercise data with other biometric information, it can more accurately estimate mental health status. For instance, it can evaluate the impact of insufficient or excessive exercise on mental health and provide appropriate exercise guidance. It can also analyze the timing and duration of exercise to find patterns related to mental health. This allows for a comprehensive understanding of mental health status using exercise data and enables appropriate responses.

[0079] The mental health assessment system can further collect sleep environment data from subjects and estimate their mental health status. For example, it can analyze bedroom temperature, humidity, and lighting brightness to identify factors that influence mental health. By combining sleep environment data with other biometric information, it can more accurately estimate mental health status. For instance, it can evaluate the impact of improving the sleep environment on mental health and make appropriate environmental adjustments. It can also analyze changes in the sleep environment to find patterns related to mental health. This allows for a comprehensive understanding of mental health status using sleep environment data and enables appropriate responses.

[0080] The mental health assessment system can further collect data on the subject's social activities and estimate their mental health status. For example, it can analyze the frequency and content of interactions with friends and family to identify factors that influence mental health. By combining social activity data with other biometric information, it can more accurately estimate mental health status. For instance, it can assess the impact of loneliness and social stress on mental health and provide appropriate social support. It can also analyze changes in social activities to identify patterns related to mental health. This allows for a comprehensive understanding of mental health status using social activity data and enables appropriate responses.

[0081] The mental health assessment system can further collect data on the subject's hobbies and interests to estimate their mental health status. For example, it can analyze the type, frequency, and changes in interests of hobbies to identify factors that influence mental health. By combining data on hobbies and interests with other biometric information, it can more accurately estimate mental health. For instance, it can evaluate the impact of decreased hobby activities or loss of interest on mental health and suggest appropriate hobby activities. It can also analyze changes in hobbies and interests to find patterns related to mental health. This allows for a comprehensive understanding of mental health using data on hobbies and interests, enabling appropriate responses.

[0082] The following briefly describes the processing flow for example form 1.

[0083] Step 1: The facial recognition unit analyzes the subject's complexion and facial expressions. For example, it detects changes in complexion and subtle changes in facial expressions, and analyzes changes in hue and brightness, eyebrow movements, and the ups and downs of the corners of the mouth. Step 2: The speech recognition unit analyzes the tone and volume of the subject's voice. For example, it detects changes in voice tone and volume, and analyzes changes in pitch, intonation, decibels, and sound pressure. Step 3: The wearable device unit collects the subject's biometric information. For example, it collects data such as heart rate, body temperature, and activity level, and analyzes the range and variability patterns of heart rate, the range and variability patterns of body temperature, steps taken, and exercise intensity. Step 4: The camera analysis unit analyzes the subject's behavior. For example, it detects changes in walking style and posture, and analyzes stride length, walking speed, degree of back straightness, and shoulder position. Step 5: The integrated analysis unit comprehensively analyzes the data collected by the facial recognition unit, voice recognition unit, wearable device unit, and camera analysis unit. For example, it comprehensively analyzes data such as changes in facial color, changes in voice tone, and changes in heart rate to estimate the mental health state. It estimates the mental health state using data weighting and analysis algorithms. Step 6: The counseling guidance unit guides the client to counseling based on the results analyzed by the integrated analysis unit. For example, if the client's mental health is deteriorating, the unit guides them to counseling and adjusts the timing, method, and content of the guidance.

[0084] (Example of form 2) The mental health status assessment system according to an embodiment of the present invention is a system that assesses the mental health status of a subject from the characteristics of their voice, complexion, and behavior, using facial recognition, voice recognition, wearable devices, camera solutions, etc. The mental health status assessment system can detect depression early, assess the state of mental health, and notice changes in the state that the person is unaware of. First, facial recognition technology is used to analyze the subject's complexion and facial expressions. For example, changes in complexion and subtle changes in facial expressions are detected, and the mental health status is estimated based on this data. Next, voice recognition technology is used to analyze the tone and volume of the subject's voice. For example, changes in voice tone or volume can be detected to indicate a change in mental health status. Furthermore, wearable devices are used to collect biometric information from the subject. For example, data such as heart rate, body temperature, and activity level are collected, and the mental health status is estimated based on this data. It is also possible to analyze the subject's behavior using a camera solution. For example, changes in walking style and posture are detected, and the mental health status is estimated based on this data. This data is comprehensively analyzed by AI, allowing for a complete understanding of the subject's mental health status. For example, if changes in facial color, voice tone, and heart rate are detected simultaneously, it is determined that there is a high probability of depression. In this way, the AI ​​can monitor the subject's mental health in real time and take appropriate action, such as guiding them to counseling, as needed. This system allows subjects to accurately understand their own mental health and take appropriate action. For example, early detection of depression allows for early initiation of treatment. Also, by noticing changes in mental health, they can re-evaluate their stress management and relaxation methods. This makes it possible to live in a state of constant mental and physical health. In this way, the mental health monitoring system can comprehensively understand the subject's mental health and take appropriate action.

[0085] The mental health status assessment system according to this embodiment comprises a facial recognition unit, a voice recognition unit, a wearable device unit, a camera analysis unit, an integrated analysis unit, and a counseling guidance unit. The facial recognition unit analyzes the subject's complexion and facial expressions. The facial recognition unit detects, for example, changes in complexion and subtle changes in facial expressions. To detect changes in complexion, the facial recognition unit can analyze changes in hue and brightness. To detect subtle changes in facial expressions, the facial recognition unit can analyze eyebrow movements and the raising and lowering of the corners of the mouth. The voice recognition unit analyzes the subject's voice tone and volume. The voice recognition unit detects, for example, changes in voice tone and volume. To detect changes in voice tone, the voice recognition unit can analyze changes in pitch and intonation. To detect changes in voice volume, the voice recognition unit can analyze changes in decibels and sound pressure. The wearable device unit collects the subject's biometric information. The wearable device unit collects data such as heart rate, body temperature, and activity level. The wearable device unit can analyze the range and variability patterns of heart rate in order to collect heart rate data. The wearable device unit can analyze the range and variability patterns of body temperature in order to collect body temperature data. The wearable device unit can analyze steps taken and exercise intensity in order to collect activity levels. The camera analysis unit analyzes the behavior of the subject. For example, the camera analysis unit detects changes in walking style and posture. For detecting walking style, the camera analysis unit can analyze stride length and walking speed. For detecting changes in posture, the camera analysis unit can analyze the degree of spinal straightness and shoulder position. The integrated analysis unit comprehensively analyzes the data collected by the facial recognition unit, voice recognition unit, wearable device unit, and camera analysis unit. For example, the integrated analysis unit comprehensively analyzes data such as changes in facial color, changes in voice tone, and changes in heart rate to estimate mental health status. The integrated analysis unit can estimate mental health status using data weighting and analysis algorithms. The counseling guidance unit guides the user to counseling based on the results analyzed by the integrated analysis unit. For example, the counseling guidance unit guides the user to counseling if their mental health is deteriorating.The counseling guidance unit can adjust the timing, method, and content of the guidance. As a result, the mental health status assessment system according to this embodiment can comprehensively understand the mental health status of the subject and take appropriate action.

[0086] The facial recognition unit analyzes the subject's complexion and facial expressions. Specifically, the facial recognition unit uses a high-resolution camera to capture the subject's face and analyzes the video data in real time. To detect changes in complexion, it can analyze changes in hue and brightness. For example, increased redness in the complexion may indicate stress or excitement, while a pale complexion may indicate fatigue or anxiety. To detect these changes, the facial recognition unit analyzes fluctuations in RGB values ​​and uses an algorithm to identify specific patterns. Furthermore, to detect subtle changes in facial expression, it can analyze eyebrow movements and the ups and downs of the corners of the mouth. For example, furrowed brows may indicate anxiety or tension, while drooping corners of the mouth may indicate sadness or depression. To detect these changes in facial expression, the facial recognition unit tracks the movement of each part of the face and analyzes subtle changes with high accuracy. In addition, the facial recognition unit can accumulate this data and monitor long-term changes. This makes it possible to grasp trends in the subject's mental health and detect abnormalities early.

[0087] The speech recognition unit analyzes the tone and volume of the subject's voice. Specifically, the speech recognition unit uses a high-sensitivity microphone to collect the subject's voice and analyzes the audio data in real time. To detect changes in voice tone, it can analyze changes in pitch and intonation. For example, a higher voice tone may indicate excitement or tension, while a lower tone may indicate calmness or fatigue. To detect these changes, the speech recognition unit analyzes the frequency components of the voice waveform and uses an algorithm to identify specific patterns. Furthermore, to detect changes in voice volume, it can analyze changes in decibels and sound pressure. For example, a rapid change in voice volume may indicate emotional fluctuations, while maintaining a constant volume may indicate a stable state. To detect these changes, the speech recognition unit analyzes the amplitude of the voice and uses an algorithm to identify specific patterns. In addition, the speech recognition unit can accumulate this data and monitor long-term changes. This makes it possible to grasp trends in the subject's mental health and detect abnormalities early.

[0088] The wearable device unit collects biometric information from the subject. Specifically, the wearable device unit uses a device worn by the subject to collect data such as heart rate, body temperature, and activity level. To collect heart rate, a heart rate sensor can be used to analyze the range and fluctuation patterns of the heart rate. For example, a rapid increase in heart rate may indicate stress or excitement, while a stable heart rate within a certain range may indicate relaxation. To detect these changes, the wearable device unit analyzes time-series data of heart rate and uses an algorithm to identify specific patterns. Similarly, to collect body temperature, a body temperature sensor can be used to analyze the range and fluctuation patterns of body temperature. For example, an increase in body temperature may indicate fever or stress, while a stable body temperature within a certain range may indicate a healthy state. To detect these changes, the wearable device unit analyzes time-series data of body temperature and uses an algorithm to identify specific patterns. Furthermore, to collect activity level, an accelerometer and gyroscope can be used to analyze steps taken and exercise intensity. For example, an increase in activity level may indicate an active state, while a decrease may indicate fatigue or depression. To detect these changes, the wearable device analyzes time-series data of activity levels and uses an algorithm to identify specific patterns. This allows the wearable device to collect biometric information of the subject with high accuracy and comprehensively understand their mental health status.

[0089] The camera analysis unit analyzes the behavior of the subject. Specifically, the camera analysis unit uses a high-resolution camera to capture the subject's movements and analyzes the video data in real time. To detect walking patterns, it can analyze stride length and walking speed. For example, a shorter stride and slower walking speed may indicate fatigue or depression, while a longer stride and faster walking speed may indicate an active state. To detect these changes, the camera analysis unit tracks the subject's movements from the video data and uses an algorithm to identify specific patterns. Furthermore, to detect changes in posture, it can analyze the degree of spinal straightness and shoulder position. For example, a curved spine and drooping shoulders may indicate fatigue or depression, while a straight spine and raised shoulders may indicate confidence and vitality. To detect these changes, the camera analysis unit analyzes the subject's posture from the video data and uses an algorithm to identify specific patterns. In addition, the camera analysis unit can accumulate this data and monitor long-term changes. This makes it possible to grasp trends in the subject's mental health and detect abnormalities early.

[0090] The integrated analysis unit comprehensively analyzes data collected by the facial recognition unit, voice recognition unit, wearable device unit, and camera analysis unit. Specifically, the integrated analysis unit centrally manages the data collected from each unit and estimates mental health status using data weighting and analysis algorithms. For example, it comprehensively analyzes data such as changes in facial color, changes in voice tone, and changes in heart rate to estimate mental health status. The integrated analysis unit analyzes this data over time, allowing it to grasp not only short-term changes but also long-term trends. Furthermore, the integrated analysis unit can use anomaly detection algorithms to detect unusual patterns and abnormal data, and issue warnings early. For example, if a sudden increase in heart rate or a sudden change in voice tone is detected, the integrated analysis unit immediately detects the anomaly and prompts appropriate action. The integrated analysis unit can also utilize historical data and statistical information to perform long-term risk assessment and trend analysis. As a result, the integrated analysis unit can not only grasp the situation in real time but also handle long-term risk management and anomaly detection, improving the reliability and safety of the entire system.

[0091] The Counseling Guidance Unit guides individuals to counseling based on the results analyzed by the Integrated Analysis Unit. Specifically, the Counseling Guidance Unit guides individuals to counseling when their mental health is deteriorating. For example, if an individual's mental health falls below a certain standard, the Counseling Guidance Unit sends a notification to the individual prompting them to schedule a counseling session. This notification is sent via a smartphone app, email, or SMS. The Counseling Guidance Unit can also adjust the timing, method, and content of the guidance. For example, it considers the individual's schedule and past counseling history to send a notification at the optimal time. The content of the notification is also customized according to the individual's condition. For example, it suggests relaxation methods for mild stressors and recommends a consultation with a professional counselor for severe stressors. Furthermore, the Counseling Guidance Unit collects feedback from individuals and can continuously improve the accuracy and effectiveness of the guidance. This allows the Counseling Guidance Unit to quickly provide appropriate counseling guidance to individuals and support the improvement of their mental health.

[0092] The facial recognition unit can detect changes in skin tone and subtle changes in facial expression. For example, to detect changes in skin tone, the facial recognition unit analyzes changes in hue and brightness. To detect subtle changes in facial expression, the facial recognition unit can analyze eyebrow movements and the ups and downs of the corners of the mouth. By analyzing changes in skin tone and facial expression in detail, it is possible to estimate mental health more accurately. Some or all of the above processing in the facial recognition unit may be performed using AI, for example, or without AI. For example, the facial recognition unit can perform analysis using generative AI to detect changes in skin tone and subtle changes in facial expression.

[0093] The speech recognition unit can detect changes in voice tone and volume. For example, to detect changes in voice tone, the speech recognition unit can analyze changes in pitch and intonation. To detect changes in voice volume, the speech recognition unit can analyze changes in decibels and sound pressure. By analyzing changes in voice tone and volume in detail, it is possible to estimate mental health status more accurately. Some or all of the above processing in the speech recognition unit may be performed using AI, for example, or without AI. For example, the speech recognition unit can perform analysis using generative AI to detect changes in voice tone and volume.

[0094] The wearable device can collect biometric information such as heart rate, body temperature, and activity level. For example, to collect heart rate, the wearable device can analyze the range and fluctuation patterns of heart rate. To collect body temperature, the wearable device can analyze the range and fluctuation patterns of body temperature. To collect activity level, the wearable device can analyze steps taken and exercise intensity. By collecting biometric information in this way, the mental health state can be estimated more accurately. Some or all of the above processing in the wearable device may be performed using AI, for example, or without AI. For example, to collect biometric information such as heart rate, body temperature, and activity level, the wearable device can perform analysis using generative AI.

[0095] The camera analysis unit can detect changes in walking style and posture. For example, to detect walking style, the camera analysis unit can analyze stride length and walking speed. To detect changes in posture, the camera analysis unit can analyze the degree of spinal straightness and shoulder position. By analyzing changes in behavior in detail, it is possible to estimate mental health status more accurately. Some or all of the above processing in the camera analysis unit may be performed using AI, for example, or without AI. For example, the camera analysis unit can perform analysis using generative AI to detect changes in walking style and posture.

[0096] The integrated analysis unit can comprehensively analyze data such as changes in facial color, voice tone, and heart rate to estimate mental health status. The integrated analysis unit estimates mental health status using, for example, data weighting and analysis algorithms. This allows for a more accurate estimation of mental health status by comprehensively analyzing multiple data. Some or all of the above-described processes in the integrated analysis unit may be performed using, for example, AI, or not. For example, the integrated analysis unit can use generative AI to perform analysis in order to comprehensively analyze data such as changes in facial color, voice tone, and heart rate.

[0097] The counseling guidance unit can guide the user to counseling as needed, based on the results analyzed by the integrated analysis unit. For example, the counseling guidance unit will guide the user to counseling if their mental health is deteriorating. The counseling guidance unit can adjust the timing, method, and content of the guidance. This allows for improvement of the user's mental health by providing appropriate guidance to counseling based on the analysis results. Some or all of the above-described processes in the counseling guidance unit may be performed using AI, for example, or without AI. For example, the counseling guidance unit can input the results analyzed by the integrated analysis unit into a generating AI and have the generating AI execute a method for guiding the user to counseling.

[0098] The facial recognition unit can estimate the subject's emotions and analyze changes in facial color in more detail based on the estimated emotions. For example, if the subject is stressed, the facial recognition unit can analyze changes in redness or blueness of the facial color in detail. If the subject is relaxed, the facial recognition unit can analyze the uniformity and healthiness of the facial color in detail. If the subject is tense, the facial recognition unit can analyze subtle changes in facial color and blood flow in detail. This allows for a more accurate estimation of mental health by analyzing changes in facial color in detail based on emotions. Some or all of the above processing in the facial recognition unit may be performed using AI, for example, or without AI. For example, the facial recognition unit can use generative AI to estimate the subject's emotions and analyze changes in facial color in detail based on the estimated emotions.

[0099] The facial recognition unit can analyze the moisture content and blood flow of the subject's skin during facial recognition and estimate their mental health. For example, the facial recognition unit can analyze the moisture content of the subject's skin and evaluate its dryness or degree of moisture. The facial recognition unit can analyze the subject's blood flow and evaluate its state of poor or improved blood circulation. The facial recognition unit can combine changes in the subject's skin moisture content and blood flow to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing skin moisture content and blood flow. Some or all of the above processing in the facial recognition unit may be performed using AI, for example, or without AI. For example, the facial recognition unit can use generative AI to analyze the moisture content and blood flow of the subject's skin during facial recognition.

[0100] The facial recognition unit can analyze the subject's eye movements and blinking frequency during facial recognition to estimate their mental health. For example, the facial recognition unit can analyze the subject's eye movements and evaluate the movement of their gaze and how well they focus. The facial recognition unit can analyze the subject's blinking frequency to evaluate the degree of tension and fatigue. By combining the subject's eye movements and blinking frequency, the facial recognition unit can comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing eye movements and blinking frequency. Some or all of the above-described processes in the facial recognition unit may be performed using AI, for example, or without AI. For example, the facial recognition unit can use generative AI to analyze the subject's eye movements and blinking frequency during facial recognition.

[0101] The facial recognition unit can estimate the subject's emotions and analyze changes in facial expression in real time based on the estimated emotions. For example, if the subject is happy, the facial recognition unit can analyze the degree of their smile and the brightness of their eyes in real time. If the subject is sad, the facial recognition unit can analyze the degree to which the corners of their mouth are turned down and the moisture in their eyes in real time. If the subject is surprised, the facial recognition unit can analyze the degree to which their eyes are open and the movement of their eyebrows in real time. This allows for a more accurate estimation of mental health by analyzing changes in facial expression in real time based on emotions. Some or all of the above processing in the facial recognition unit may be performed using AI, for example, or without AI. For example, the facial recognition unit can use generative AI to estimate the subject's emotions and analyze changes in facial expression in real time based on the estimated emotions.

[0102] The facial recognition unit can analyze the temperature distribution of a subject's face during facial recognition and estimate their mental health. For example, the facial recognition unit can analyze the temperature distribution of a subject's face and evaluate areas that tend to get cold easily and areas that tend to get warm easily. The facial recognition unit can analyze changes in the temperature distribution of a subject's face and evaluate their stress and relaxation levels. The facial recognition unit can combine the temperature distribution of a subject's face with other biometric information to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing the temperature distribution of the face. Some or all of the above-described processes in the facial recognition unit may be performed using AI, for example, or without AI. For example, the facial recognition unit can use generative AI to analyze the temperature distribution of a subject's face during facial recognition.

[0103] The facial recognition unit can analyze the tension of the subject's facial muscles during facial recognition and estimate their mental health. For example, the facial recognition unit can analyze the tension of the subject's facial muscles and evaluate whether they are relaxed or tense. The facial recognition unit can analyze changes in the tension of the subject's facial muscles and evaluate the degree of stress or fatigue. The facial recognition unit can combine the tension of the subject's facial muscles with other biometric information to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing the tension of facial muscles. Some or all of the above-described processes in the facial recognition unit may be performed using AI, for example, or without AI. For example, the facial recognition unit can use generative AI to analyze the tension of the subject's facial muscles during facial recognition.

[0104] The speech recognition unit can estimate the subject's emotions and analyze changes in voice tone in more detail based on the estimated emotions. For example, if the subject is nervous, the speech recognition unit can analyze the pitch and tremor of the voice tone in detail. If the subject is relaxed, the speech recognition unit can analyze the stability and smoothness of the voice tone in detail. If the subject is excited, the speech recognition unit can analyze the rapid changes and intensity of the voice tone in detail. This allows for a more accurate estimation of mental health by analyzing changes in voice tone in detail based on emotions. Some or all of the above processing in the speech recognition unit may be performed using AI, for example, or without AI. For example, the speech recognition unit can use generative AI to estimate the subject's emotions and analyze changes in voice tone in detail based on the estimated emotions.

[0105] The speech recognition unit can analyze the subject's breathing sounds and cough frequency during speech recognition to estimate their mental health. For example, the speech recognition unit can analyze the subject's breathing sounds and evaluate the depth and rhythm of their breathing. The speech recognition unit can analyze the subject's cough frequency and evaluate the degree of stress and fatigue. The speech recognition unit can combine the subject's breathing sounds and cough frequency to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing breathing sounds and cough frequency. Some or all of the above processing in the speech recognition unit may be performed using AI, for example, or without AI. For example, the speech recognition unit can use generative AI to analyze the subject's breathing sounds and cough frequency during speech recognition.

[0106] The speech recognition unit can analyze the speed and rhythm of a person's speech during speech recognition and estimate their mental health. For example, the speech recognition unit can analyze the speed of the person's speech and evaluate the degree of tension or relaxation. The speech recognition unit can analyze the rhythm of the person's speech and evaluate the degree of stress or fatigue. The speech recognition unit can combine the speed and rhythm of the person's speech to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing the speed and rhythm of speech. Some or all of the above processing in the speech recognition unit may be performed using AI, for example, or without AI. For example, the speech recognition unit can use generative AI to analyze the speed and rhythm of the person's speech during speech recognition.

[0107] The speech recognition unit can estimate the subject's emotions and analyze changes in voice volume in real time based on the estimated emotions. For example, if the subject is nervous, the speech recognition unit can analyze changes in voice volume in real time. If the subject is relaxed, the speech recognition unit can analyze the stability of voice volume in real time. If the subject is excited, the speech recognition unit can analyze rapid changes in voice volume in real time. This allows for a more accurate estimation of mental health by analyzing changes in voice volume in real time based on emotions. Some or all of the above processing in the speech recognition unit may be performed using AI, for example, or without AI. For example, the speech recognition unit can use generative AI to estimate the subject's emotions and analyze changes in voice volume in real time based on the estimated emotions.

[0108] The speech recognition unit can analyze the frequency components of a subject's voice during speech recognition and estimate their mental health. For example, the speech recognition unit can analyze the frequency components of a subject's voice and evaluate the degree of tension or relaxation. The speech recognition unit can analyze changes in the frequency components of a subject's voice and evaluate the degree of stress or fatigue. The speech recognition unit can combine the frequency components of a subject's voice with other biometric information to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing the frequency components of the voice. Some or all of the above-described processes in the speech recognition unit may be performed using AI, for example, or without AI. For example, the speech recognition unit can use generative AI to analyze the frequency components of a subject's voice during speech recognition.

[0109] The speech recognition unit can analyze the intonation and pitch of a subject's voice during speech recognition to estimate their mental health. For example, the speech recognition unit can analyze the intonation of a subject's voice to evaluate the degree of tension or relaxation. The speech recognition unit can analyze the pitch of a subject's voice to evaluate the degree of stress or fatigue. The speech recognition unit can combine the intonation and pitch of a subject's voice to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing the intonation and pitch of the voice. Some or all of the above processing in the speech recognition unit may be performed using AI, for example, or without AI. For example, the speech recognition unit can use generative AI to analyze the intonation and pitch of a subject's voice during speech recognition.

[0110] The wearable device can estimate the subject's emotions and analyze changes in heart rate in more detail based on the estimated emotions. For example, if the subject is tense, the wearable device can analyze rapid changes in heart rate in detail. If the subject is relaxed, the wearable device can analyze the stability of heart rate in detail. If the subject is excited, the wearable device can analyze rapid increases in heart rate in detail. This allows for a more accurate estimation of mental health by analyzing changes in heart rate in detail based on emotions. Some or all of the above processing in the wearable device may be performed using AI, for example, or without AI. For example, the wearable device can use generative AI to estimate the subject's emotions and analyze changes in heart rate in detail based on the estimated emotions.

[0111] The wearable device unit can analyze a subject's blood pressure and blood glucose levels to estimate their mental health. For example, the wearable device unit can analyze a subject's blood pressure to assess their stress and relaxation levels. The wearable device unit can analyze a subject's blood glucose levels to assess their fatigue and energy levels. The wearable device unit can combine the subject's blood pressure and blood glucose levels to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing blood pressure and blood glucose levels. Some or all of the above-described processes in the wearable device unit may be performed using AI, for example, or without AI. For example, the wearable device unit can use generative AI to analyze a subject's blood pressure and blood glucose levels.

[0112] The wearable device unit can analyze a subject's sleep patterns and estimate their mental health. For example, the wearable device unit can analyze a subject's sleep patterns and evaluate the quality and quantity of sleep. The wearable device unit can analyze changes in a subject's sleep patterns and evaluate the degree of stress and fatigue. The wearable device unit can combine a subject's sleep patterns with other biometric information to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing sleep patterns. Some or all of the above-described processes in the wearable device unit may be performed using AI, for example, or without AI. For example, the wearable device unit can use generative AI to analyze a subject's sleep patterns.

[0113] The wearable device can estimate the subject's emotions and analyze changes in body temperature in real time based on the estimated emotions. For example, if the subject is tense, the wearable device can analyze rapid changes in body temperature in real time. If the subject is relaxed, the wearable device can analyze the stability of body temperature in real time. If the subject is excited, the wearable device can analyze rapid increases in body temperature in real time. This allows for a more accurate estimation of mental health by analyzing changes in body temperature in real time based on emotions. Some or all of the above processing in the wearable device may be performed using AI, for example, or without AI. For example, the wearable device can use generative AI to estimate the subject's emotions and analyze changes in body temperature in real time based on the estimated emotions.

[0114] The wearable device unit can analyze the subject's skin electrical activity and estimate their mental health state. For example, the wearable device unit can analyze the subject's skin electrical activity and evaluate the degree of stress and relaxation. The wearable device unit can analyze changes in the subject's skin electrical activity and evaluate the degree of tension and fatigue. The wearable device unit can combine the subject's skin electrical activity with other biometric information to comprehensively estimate their mental health state. This allows for a more accurate estimation of mental health state by analyzing skin electrical activity. Some or all of the above-described processes in the wearable device unit may be performed using AI, for example, or without AI. For example, the wearable device unit can use generative AI to analyze the subject's skin electrical activity.

[0115] The wearable device unit can analyze changes in a subject's activity level and estimate their mental health. For example, the wearable device unit can analyze a subject's activity level and evaluate their daily exercise and activity patterns. The wearable device unit can analyze changes in a subject's activity level and evaluate their stress and fatigue levels. The wearable device unit can combine a subject's activity level with other biometric information to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing changes in activity level. Some or all of the above-described processes in the wearable device unit may be performed using AI, for example, or without AI. For example, the wearable device unit can use generative AI to analyze changes in a subject's activity level.

[0116] The camera analysis unit can estimate the subject's emotions and analyze changes in their gait in more detail based on those estimated emotions. For example, if the subject is tense, the camera analysis unit can analyze changes in walking speed and rhythm in detail. If the subject is relaxed, the camera analysis unit can analyze the stability and smoothness of their gait in detail. If the subject is excited, the camera analysis unit can analyze sudden changes and variations in gait in detail. This allows for a more accurate estimation of mental health by analyzing changes in gait based on emotions. Some or all of the above processing in the camera analysis unit may be performed using AI, for example, or without AI. For example, the camera analysis unit can use generative AI to estimate the subject's emotions and analyze changes in their gait in detail based on those estimated emotions.

[0117] The camera analysis unit can analyze subtle changes in a subject's posture during camera analysis and estimate their mental health. For example, the camera analysis unit can analyze subtle changes in a subject's posture and evaluate their level of tension or relaxation. The camera analysis unit can analyze changes in a subject's posture and evaluate their level of stress or fatigue. The camera analysis unit can combine subtle changes in a subject's posture with other biometric information to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing subtle changes in posture. Some or all of the above-described processes in the camera analysis unit may be performed using AI, for example, or without AI. For example, the camera analysis unit can use generative AI to analyze subtle changes in a subject's posture during camera analysis.

[0118] The camera analysis unit can analyze the subject's hand movements and gestures during camera analysis to estimate their mental health. For example, the camera analysis unit can analyze the subject's hand movements to evaluate their level of tension or relaxation. The camera analysis unit can analyze the subject's gestures to evaluate their level of stress or fatigue. The camera analysis unit can combine the subject's hand movements and gestures to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing hand movements and gestures. Some or all of the above processing in the camera analysis unit may be performed using AI, for example, or without AI. For example, the camera analysis unit can use generative AI to analyze the subject's hand movements and gestures during camera analysis.

[0119] The camera analysis unit can estimate the subject's emotions and analyze changes in posture in real time based on the estimated emotions. For example, if the subject is tense, the camera analysis unit can analyze changes in posture in real time. If the subject is relaxed, the camera analysis unit can analyze the stability of their posture in real time. If the subject is excited, the camera analysis unit can analyze rapid changes in posture in real time. This allows for a more accurate estimation of mental health by analyzing changes in posture in real time based on emotions. Some or all of the above processing in the camera analysis unit may be performed using AI, for example, or without AI. For example, the camera analysis unit can use generative AI to estimate the subject's emotions and analyze changes in posture in real time based on the estimated emotions.

[0120] The camera analysis unit can analyze the speed and rhythm of a subject's movements during camera analysis and estimate their mental health. For example, the camera analysis unit can analyze the speed of a subject's movements and evaluate their level of tension or relaxation. The camera analysis unit can analyze the rhythm of a subject's movements and evaluate their level of stress or fatigue. The camera analysis unit can combine the speed and rhythm of a subject's movements to comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing the speed and rhythm of movements. Some or all of the above-described processes in the camera analysis unit may be performed using AI, for example, or without AI. For example, the camera analysis unit can use generative AI to analyze the speed and rhythm of a subject's movements during camera analysis.

[0121] The camera analysis unit can analyze the consistency and coordination of a subject's movements during camera analysis and estimate their mental health. For example, the camera analysis unit can analyze the consistency of a subject's movements and evaluate their level of tension and relaxation. The camera analysis unit can analyze the coordination of a subject's movements and evaluate their level of stress and fatigue. By combining the consistency and coordination of a subject's movements, the camera analysis unit can comprehensively estimate their mental health. This allows for a more accurate estimation of mental health by analyzing the consistency and coordination of movements. Some or all of the above-described processes in the camera analysis unit may be performed using AI, for example, or without AI. For example, the camera analysis unit can use generative AI to analyze the consistency and coordination of a subject's movements during camera analysis.

[0122] The integrated analysis unit can estimate the subject's emotions and adjust the integrated analysis algorithm based on the estimated emotions. For example, if the subject is tense, the integrated analysis unit will prioritize data related to tension in its analysis. If the subject is relaxed, the integrated analysis unit can prioritize data related to relaxation in its analysis. If the subject is excited, the integrated analysis unit can prioritize data related to excitement in its analysis. By adjusting the integrated analysis algorithm based on emotions, the mental health state can be estimated more accurately. Some or all of the above processing in the integrated analysis unit may be performed using AI, for example, or without AI. For example, the integrated analysis unit can use generative AI to perform analysis in order to estimate the subject's emotions and adjust the integrated analysis algorithm based on the estimated emotions.

[0123] The integrated analysis unit can predict the current mental health state by referring to past data during integrated analysis. For example, the integrated analysis unit can predict the current mental health state by referring to past data. The integrated analysis unit can predict changes in the mental health state by comparing past data with current data. The integrated analysis unit can predict the future mental health state based on past data. This allows for a more accurate prediction of the current mental health state by referring to past data. Some or all of the above processes in the integrated analysis unit may be performed using AI, for example, or without AI. For example, the integrated analysis unit can use generative AI to perform analysis in order to predict the current mental health state by referring to past data during integrated analysis.

[0124] The integrated analysis unit can analyze correlations between different data sources during integrated analysis and estimate mental health status. For example, the integrated analysis unit can analyze the correlation between changes in facial color and changes in voice tone to estimate mental health status. The integrated analysis unit can analyze the correlation between changes in heart rate and changes in gait to estimate mental health status. The integrated analysis unit can analyze the correlation between changes in skin electrical activity and changes in posture to estimate mental health status. In this way, mental health status can be estimated more accurately by analyzing correlations between different data sources. Some or all of the above processing in the integrated analysis unit may be performed using AI, for example, or without AI. For example, the integrated analysis unit can use generative AI to analyze correlations between different data sources during integrated analysis.

[0125] The integrated analysis unit can estimate the subject's emotions and adjust the order in which the results of the integrated analysis are displayed based on the estimated emotions. For example, if the subject is tense, the integrated analysis unit can prioritize displaying data related to tension. If the subject is relaxed, the integrated analysis unit can prioritize displaying data related to relaxation. If the subject is excited, the integrated analysis unit can prioritize displaying data related to excitement. By adjusting the order in which results are displayed based on emotions, the mental health state can be estimated more accurately. Some or all of the above processing in the integrated analysis unit may be performed using AI, for example, or without AI. For example, the integrated analysis unit can use generative AI to perform analysis in order to estimate the subject's emotions and adjust the order in which the results of the integrated analysis are displayed based on the estimated emotions.

[0126] The integrated analysis unit can estimate the mental health status of a subject by referring to their living environment data during integrated analysis. For example, the integrated analysis unit can estimate the mental health status by referring to the subject's living environment data. The integrated analysis unit can comprehensively estimate the mental health status by combining the subject's living environment data with other biometric information. The integrated analysis unit can predict the future mental health status based on the subject's living environment data. This allows for a more accurate estimation of mental health status by referring to living environment data. Some or all of the above-described processes in the integrated analysis unit may be performed using AI, for example, or without AI. For example, the integrated analysis unit can use generative AI to perform analysis in order to estimate the mental health status by referring to the subject's living environment data during integrated analysis.

[0127] The integrated analysis unit can estimate the mental health status of a subject by referring to their social relationship data during integrated analysis. For example, the integrated analysis unit can estimate the mental health status by referring to the subject's social relationship data. The integrated analysis unit can comprehensively estimate the mental health status by combining the subject's social relationship data with other biometric information. The integrated analysis unit can predict the future mental health status of a subject based on their social relationship data. This allows for a more accurate estimation of mental health status by referring to social relationship data. Some or all of the above-described processes in the integrated analysis unit may be performed using AI, for example, or without AI. For example, the integrated analysis unit can use generative AI to perform analysis in order to estimate the mental health status of a subject by referring to their social relationship data during integrated analysis.

[0128] The counseling guidance unit can estimate the client's emotions and adjust the counseling guidance method based on the estimated emotions. For example, if the client is tense, the counseling guidance unit can provide a guidance method that helps them relax. If the client is relaxed, the counseling guidance unit can provide detailed counseling. If the client is agitated, the counseling guidance unit can provide a guidance method that helps them calm down. In this way, by adjusting the counseling guidance method based on emotions, more appropriate counseling can be provided. Some or all of the above processing in the counseling guidance unit may be performed using AI, for example, or without AI. For example, the counseling guidance unit can use generative AI to perform analysis in order to estimate the client's emotions and adjust the counseling guidance method based on the estimated emotions.

[0129] The counseling guidance unit can select the optimal guidance method by referring to the subject's past counseling history during counseling guidance. For example, the counseling guidance unit can refer to the subject's past counseling history and select the optimal guidance method. The counseling guidance unit can combine the subject's past counseling history with current data to select the optimal guidance method. The counseling guidance unit can predict future counseling methods based on the subject's past counseling history. This allows for the provision of more appropriate counseling by referring to past counseling history. Some or all of the above processing in the counseling guidance unit may be performed using AI, for example, or without AI. For example, the counseling guidance unit can use generative AI to perform analysis in order to select the optimal guidance method by referring to the subject's past counseling history during counseling guidance.

[0130] The counseling guidance unit can estimate the client's emotions and determine the priority of counseling based on the estimated emotions. For example, if the client is tense, the counseling guidance unit may prioritize counseling that helps them relax. If the client is relaxed, the counseling guidance unit may prioritize detailed counseling. If the client is agitated, the counseling guidance unit may prioritize counseling that helps them calm down. By determining the priority of counseling based on emotions, more appropriate counseling can be provided. Some or all of the above processing in the counseling guidance unit may be performed using AI, for example, or without AI. For example, the counseling guidance unit may use generative AI to perform analysis in order to estimate the client's emotions and determine the priority of counseling based on the estimated emotions.

[0131] The counseling guidance unit can select the optimal guidance method by referring to the subject's living environment data during counseling guidance. For example, the counseling guidance unit can select the optimal guidance method by referring to the subject's living environment data. The counseling guidance unit can select the optimal guidance method by combining the subject's living environment data with other biometric information. The counseling guidance unit can predict future guidance methods based on the subject's living environment data. This allows for more appropriate counseling to be provided by referring to the living environment data. Some or all of the above processing in the counseling guidance unit may be performed using AI, for example, or without AI. For example, the counseling guidance unit can use generative AI to perform analysis in order to select the optimal guidance method by referring to the subject's living environment data during counseling guidance.

[0132] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0133] The mental health assessment system can further collect dietary data from subjects and estimate their mental health status. For example, it can analyze the content of meals, calorie intake, and nutritional balance to identify factors that influence mental health. By combining dietary data with other biometric information, it can more accurately estimate mental health status. For instance, it can evaluate the impact of nutritional deficiencies or excesses on mental health and provide appropriate dietary guidance. It can also analyze the timing and frequency of meals to find patterns related to mental health. This allows for a comprehensive understanding of mental health status using dietary data, enabling appropriate responses.

[0134] The mental health assessment system can further collect exercise data from subjects and estimate their mental health status. For example, it can analyze the type, frequency, and intensity of exercise to identify factors that influence mental health. By combining exercise data with other biometric information, it can more accurately estimate mental health status. For instance, it can evaluate the impact of insufficient or excessive exercise on mental health and provide appropriate exercise guidance. It can also analyze the timing and duration of exercise to find patterns related to mental health. This allows for a comprehensive understanding of mental health status using exercise data and enables appropriate responses.

[0135] The mental health assessment system can further collect sleep environment data from subjects and estimate their mental health status. For example, it can analyze bedroom temperature, humidity, and lighting brightness to identify factors that influence mental health. By combining sleep environment data with other biometric information, it can more accurately estimate mental health status. For instance, it can evaluate the impact of improving the sleep environment on mental health and make appropriate environmental adjustments. It can also analyze changes in the sleep environment to find patterns related to mental health. This allows for a comprehensive understanding of mental health status using sleep environment data and enables appropriate responses.

[0136] The mental health assessment system can further collect data on the subject's social activities and estimate their mental health status. For example, it can analyze the frequency and content of interactions with friends and family to identify factors that influence mental health. By combining social activity data with other biometric information, it can more accurately estimate mental health status. For instance, it can assess the impact of loneliness and social stress on mental health and provide appropriate social support. It can also analyze changes in social activities to identify patterns related to mental health. This allows for a comprehensive understanding of mental health status using social activity data and enables appropriate responses.

[0137] The mental health assessment system can further collect data on the subject's hobbies and interests to estimate their mental health status. For example, it can analyze the type, frequency, and changes in interests of hobbies to identify factors that influence mental health. By combining data on hobbies and interests with other biometric information, it can more accurately estimate mental health. For instance, it can evaluate the impact of decreased hobby activities or loss of interest on mental health and suggest appropriate hobby activities. It can also analyze changes in hobbies and interests to find patterns related to mental health. This allows for a comprehensive understanding of mental health using data on hobbies and interests, enabling appropriate responses.

[0138] The mental health assessment system can further estimate the subject's emotions and select music based on those estimated emotions. For example, if the subject is stressed, it can select relaxing music. If the subject is relaxed, it can select music to maintain that mood. If the subject is agitated, it can select calming music. In this way, selecting music based on emotions can help maintain better mental health. Music selection may or may not be performed using AI. For example, generative AI can be used to analyze emotions and select music based on those estimated emotions.

[0139] The mental health monitoring system can further estimate the subject's emotions and adjust the lighting based on those emotions. For example, if the subject is stressed, the lighting can be adjusted to a warm color to promote relaxation. If the subject is relaxed, the lighting can be adjusted to maintain that state. If the subject is agitated, the lighting can be adjusted to a calming effect. By adjusting the lighting based on emotions, the system can better maintain mental health. The lighting adjustment may be performed using AI or not. For example, generative AI can be used to analyze emotions and adjust the lighting based on those emotions.

[0140] The mental health assessment system can further estimate the subject's emotions and adjust the scent based on those emotions. For example, if the subject is stressed, it can provide a relaxing scent. If the subject is relaxed, it can provide a scent to maintain that state. If the subject is agitated, it can provide a calming scent. In this way, adjusting the scent based on emotions can help maintain better mental health. The scent adjustment may be performed using AI or not. For example, generative AI can be used to analyze emotions and adjust the scent based on those emotions.

[0141] The mental health monitoring system can further estimate the subject's emotions and adjust the temperature based on those emotions. For example, if the subject is stressed, the temperature can be adjusted to a relaxing level. If the subject is relaxed, the temperature can be adjusted to maintain that state. If the subject is agitated, the temperature can be adjusted to a calming level. In this way, adjusting the temperature based on emotions can help maintain better mental health. Temperature adjustment may be performed using AI or not. For example, generative AI can be used to analyze emotions and adjust the temperature based on those emotions.

[0142] The mental health assessment system can further estimate the subject's emotions and adjust the counseling content based on those estimated emotions. For example, if the subject is stressed, the system can provide counseling content that helps them relax. If the subject is relaxed, the system can provide counseling content that helps them maintain that state. If the subject is agitated, the system can provide counseling content that helps them calm down. In this way, by adjusting the counseling content based on emotions, it is possible to maintain better mental health. The adjustment of the counseling content may be done using AI or not. For example, generative AI can be used to analyze and estimate emotions and adjust the counseling content based on those estimated emotions.

[0143] The following briefly describes the processing flow for example form 2.

[0144] Step 1: The facial recognition unit analyzes the subject's complexion and facial expressions. For example, it detects changes in complexion and subtle changes in facial expressions, and analyzes changes in hue and brightness, eyebrow movements, and the ups and downs of the corners of the mouth. Step 2: The speech recognition unit analyzes the tone and volume of the subject's voice. For example, it detects changes in voice tone and volume, and analyzes changes in pitch, intonation, decibels, and sound pressure. Step 3: The wearable device unit collects the subject's biometric information. For example, it collects data such as heart rate, body temperature, and activity level, and analyzes the range and variability patterns of heart rate, the range and variability patterns of body temperature, steps taken, and exercise intensity. Step 4: The camera analysis unit analyzes the subject's behavior. For example, it detects changes in walking style and posture, and analyzes stride length, walking speed, degree of back straightness, and shoulder position. Step 5: The integrated analysis unit comprehensively analyzes the data collected by the facial recognition unit, voice recognition unit, wearable device unit, and camera analysis unit. For example, it comprehensively analyzes data such as changes in facial color, changes in voice tone, and changes in heart rate to estimate the mental health state. It estimates the mental health state using data weighting and analysis algorithms. Step 6: The counseling guidance unit guides the client to counseling based on the results analyzed by the integrated analysis unit. For example, if the client's mental health is deteriorating, the unit guides them to counseling and adjusts the timing, method, and content of the guidance.

[0145] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0146] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0147] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0148] Each of the multiple elements described above, including the facial recognition unit, voice recognition unit, wearable device unit, camera analysis unit, integrated analysis unit, and counseling guidance unit, is implemented in at least one of the smart device 14 and the data processing unit 12. For example, the facial recognition unit uses the camera 42 of the smart device 14 to analyze the subject's complexion and facial expressions, and the control unit 46A detects changes in complexion and subtle changes in facial expressions. The voice recognition unit uses the microphone 38B of the smart device 14 to analyze the subject's voice tone and volume, and the control unit 46A detects changes in voice tone and volume. The wearable device unit works in conjunction with the smart device 14 to collect biometric information such as heart rate, body temperature, and activity level. The camera analysis unit uses the camera 42 of the smart device 14 to analyze the subject's behavior, and the control unit 46A detects changes in walking style and posture. The integrated analysis unit uses the identification processing unit 290 of the data processing unit 12 to comprehensively analyze this data and estimate the subject's mental health status. The counseling guidance unit, using the specific processing unit 290 of the data processing device 12, guides the user to counseling based on the analysis results. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0149] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0150] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0151] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0152] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0153] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0154] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0155] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0156] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0157] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0158] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0159] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0160] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0161] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0162] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0163] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0164] Each of the multiple elements described above, including the facial recognition unit, voice recognition unit, wearable device unit, camera analysis unit, integrated analysis unit, and counseling guidance unit, is implemented in at least one of the smart glasses 214 and the data processing unit 12. For example, the facial recognition unit uses the camera 42 of the smart glasses 214 to analyze the subject's complexion and facial expressions, and the control unit 46A detects changes in complexion and subtle changes in facial expressions. The voice recognition unit uses the microphone 238 of the smart glasses 214 to analyze the subject's voice tone and volume, and the control unit 46A detects changes in voice tone and volume. The wearable device unit works in conjunction with the smart glasses 214 to collect biometric information such as heart rate, body temperature, and activity level. The camera analysis unit uses the camera 42 of the smart glasses 214 to analyze the subject's behavior, and the control unit 46A detects changes in walking style and posture. The integrated analysis unit uses the identification processing unit 290 of the data processing unit 12 to comprehensively analyze this data and estimate the subject's mental health status. The counseling guidance unit, using the specific processing unit 290 of the data processing device 12, guides the user to counseling based on the analysis results. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0165] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0166] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0167] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0168] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0169] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0170] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0171] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0172] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0173] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0174] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0175] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0176] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0177] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0178] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0179] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0180] Each of the multiple elements described above, including the facial recognition unit, voice recognition unit, wearable device unit, camera analysis unit, integrated analysis unit, and counseling guidance unit, is implemented in at least one of the following: a headset terminal 314 and a data processing unit 12. For example, the facial recognition unit uses the camera 42 of the headset terminal 314 to analyze the subject's complexion and facial expressions, and the control unit 46A detects changes in complexion and subtle changes in facial expressions. The voice recognition unit uses the microphone 238 of the headset terminal 314 to analyze the subject's voice tone and volume, and the control unit 46A detects changes in voice tone and volume. The wearable device unit works in conjunction with the headset terminal 314 to collect biometric information such as heart rate, body temperature, and activity level. The camera analysis unit uses the camera 42 of the headset terminal 314 to analyze the subject's behavior, and the control unit 46A detects changes in walking style and posture. The integrated analysis unit uses the identification processing unit 290 of the data processing unit 12 to comprehensively analyze this data and estimate the subject's mental health status. The counseling guidance unit, using the specific processing unit 290 of the data processing device 12, guides the user to counseling based on the analysis results. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0181] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0182] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0183] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0184] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0185] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0186] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0187] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0188] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0189] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0190] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0191] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0192] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0193] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0194] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0195] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0196] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0197] Each of the multiple elements described above, including the facial recognition unit, voice recognition unit, wearable device unit, camera analysis unit, integrated analysis unit, and counseling guidance unit, is implemented in, for example, at least one of the robot 414 and the data processing unit 12. For example, the facial recognition unit uses the camera 42 of the robot 414 to analyze the subject's complexion and facial expressions, and the control unit 46A detects changes in complexion and subtle changes in facial expressions. The voice recognition unit uses the microphone 238 of the robot 414 to analyze the subject's voice tone and volume, and the control unit 46A detects changes in voice tone and volume. The wearable device unit works in conjunction with the robot 414 to collect biometric information such as heart rate, body temperature, and activity level. The camera analysis unit uses the camera 42 of the robot 414 to analyze the subject's behavior, and the control unit 46A detects changes in walking style and posture. The integrated analysis unit, using the identification processing unit 290 of the data processing unit 12, comprehensively analyzes this data and estimates the subject's mental health status. The counseling guidance unit, using the specific processing unit 290 of the data processing device 12, guides the user to counseling based on the analysis results. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0198] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0199] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0200] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0201] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0202] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0203] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0204] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0205] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0206] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0207] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0208] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0209] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0210] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0211] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0212] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0213] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0214] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0215] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0216] (Note 1) A facial recognition unit analyzes the subject's complexion and facial expressions, A voice recognition unit that analyzes the tone and volume of the subject's voice, A wearable device unit that collects the subject's biometric information, A camera analysis unit that analyzes the behavior of the subject, An integrated analysis unit that comprehensively analyzes the data collected by the aforementioned facial recognition unit, voice recognition unit, wearable device unit, and camera analysis unit, The system includes a counseling guidance unit that guides the user to counseling based on the results of the analysis performed by the integrated analysis unit. A system characterized by the following features. (Note 2) The aforementioned facial recognition unit is It detects changes in skin tone and subtle changes in facial expression. The system described in Appendix 1, characterized by the features described herein. (Note 3) The aforementioned speech recognition unit, Detects changes in voice tone and volume. The system described in Appendix 1, characterized by the features described herein. (Note 4) The wearable device section is, It collects biometric information such as heart rate, body temperature, and activity level. The system described in Appendix 1, characterized by the features described herein. (Note 5) The camera analysis unit, Detects changes in walking style and posture. The system described in Appendix 1, characterized by the features described herein. (Note 6) The integrated analysis unit, By comprehensively analyzing data such as changes in facial color, voice tone, and heart rate, we estimate the state of mental health. The system described in Appendix 1, characterized by the features described herein. (Note 7) The aforementioned counseling guidance unit is Based on the results of the integrated analysis, counseling will be provided as needed. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned facial recognition unit is The system estimates the subject's emotions and then analyzes changes in facial color in more detail based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned facial recognition unit is During facial recognition, the system analyzes the subject's skin moisture content and blood flow to estimate their mental health. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned facial recognition unit is During facial recognition, the system analyzes the subject's eye movements and blinking frequency to estimate their mental health. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned facial recognition unit is It estimates the subject's emotions and analyzes changes in facial expressions in real time based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned facial recognition unit is During facial recognition, the temperature distribution of the subject's face is analyzed to estimate their mental health status. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned facial recognition unit is During facial recognition, the system analyzes the tension level of the subject's facial muscles to estimate their mental health. The system described in Appendix 1, characterized by the features described herein. (Note 14) The aforementioned speech recognition unit, The system estimates the subject's emotions and then analyzes changes in voice tone in more detail based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 15) The aforementioned speech recognition unit, During speech recognition, the system analyzes the subject's breathing sounds and cough frequency to estimate their mental health status. The system described in Appendix 1, characterized by the features described herein. (Note 16) The aforementioned speech recognition unit, During speech recognition, the system analyzes the speaker's speaking speed and rhythm to estimate their mental health. The system described in Appendix 1, characterized by the features described herein. (Note 17) The aforementioned speech recognition unit, It estimates the emotions of the subject and analyzes changes in voice volume in real time based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 18) The aforementioned speech recognition unit, During speech recognition, the frequency components of the subject's voice are analyzed to estimate their mental health status. The system described in Appendix 1, characterized by the features described herein. (Note 19) The aforementioned speech recognition unit, During speech recognition, the system analyzes the intonation and pitch of the subject's voice to estimate their mental health. The system described in Appendix 1, characterized by the features described herein. (Note 20) The wearable device section is, The system estimates the emotions of the subject and then analyzes changes in heart rate in more detail based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 21) The wearable device section is, Wearable devices analyze a person's blood pressure and blood sugar levels to estimate their mental health. The system described in Appendix 1, characterized by the features described herein. (Note 22) The wearable device section is, Wearable devices are used to analyze a subject's sleep patterns and estimate their mental health status. The system described in Appendix 1, characterized by the features described herein. (Note 23) The wearable device section is, It estimates the emotions of the subject and analyzes changes in body temperature in real time based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 24) The wearable device section is, Wearable devices analyze the electrical activity of the subject's skin to estimate their mental health. The system described in Appendix 1, characterized by the features described herein. (Note 25) The wearable device section is, Wearable devices are used to analyze changes in a subject's activity level and estimate their mental health status. The system described in Appendix 1, characterized by the features described herein. (Note 26) The camera analysis unit, The system estimates the emotions of the subjects and analyzes changes in their walking patterns in more detail based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 27) The camera analysis unit, During camera analysis, subtle changes in the subject's posture are analyzed to estimate their mental health. The system described in Appendix 1, characterized by the features described herein. (Note 28) The camera analysis unit, During camera analysis, the subject's hand movements and gestures are analyzed to estimate their mental health. The system described in Appendix 1, characterized by the features described herein. (Note 29) The camera analysis unit, It estimates the emotions of the subject and analyzes changes in posture in real time based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 30) The camera analysis unit, During camera analysis, the speed and rhythm of the subject's movements are analyzed to estimate their mental health. The system described in Appendix 1, characterized by the features described herein. (Note 31) The camera analysis unit, During camera analysis, the consistency and coordination of the subject's movements are analyzed to estimate their mental health. The system described in Appendix 1, characterized by the features described herein. (Note 32) The integrated analysis unit, The system estimates the emotions of the subjects and adjusts the integrated analysis algorithm based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 33) The integrated analysis unit, During integrated analysis, past data is referenced to predict current mental health status. The system described in Appendix 1, characterized by the features described herein. (Note 34) The integrated analysis unit, During integrated analysis, correlations between different data sources are analyzed to estimate mental health status. The system described in Appendix 1, characterized by the features described herein. (Note 35) The integrated analysis unit, The system estimates the emotions of the subjects and adjusts the order in which the results of the integrated analysis are displayed based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 36) The integrated analysis unit, During integrated analysis, the mental health status of the subjects is estimated by referring to their living environment data. The system described in Appendix 1, characterized by the features described herein. (Note 37) The integrated analysis unit, During integrated analysis, the mental health status of the subjects is estimated by referring to their social relationship data. The system described in Appendix 1, characterized by the features described herein. (Note 38) The aforementioned counseling guidance unit is We estimate the subject's emotions and adjust the counseling guidance method based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 39) The aforementioned counseling guidance unit is When guiding the client into counseling, the most appropriate guidance method is selected by referring to the client's past counseling history. The system described in Appendix 1, characterized by the features described herein. (Note 40) The aforementioned counseling guidance unit is The client's emotions are estimated, and the priority of counseling is determined based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 41) The aforementioned counseling guidance unit is During the counseling session, the most appropriate guidance method is selected by referring to the subject's living environment data. The system described in Appendix 1, characterized by the features described herein. [Explanation of Symbols]

[0217] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. A facial recognition unit analyzes the subject's complexion and facial expressions, A voice recognition unit that analyzes the tone and volume of the subject's voice, A wearable device unit that collects the subject's biometric information, A camera analysis unit that analyzes the behavior of the subject, An integrated analysis unit that comprehensively analyzes the data collected by the facial recognition unit, the voice recognition unit, the wearable device unit, and the camera analysis unit, The system includes a counseling guidance unit that guides the user to counseling based on the results of the analysis performed by the integrated analysis unit. A system characterized by the following features.

2. The aforementioned facial recognition unit is It detects changes in skin tone and subtle changes in facial expression. The system according to feature 1.

3. The aforementioned speech recognition unit, Detects changes in voice tone and volume. The system according to feature 1.

4. The wearable device section is, It collects biometric information such as heart rate, body temperature, and activity level. The system according to feature 1.

5. The camera analysis unit, Detects changes in walking style and posture. The system according to feature 1.

6. The integrated analysis unit, By comprehensively analyzing data such as changes in facial color, voice tone, and heart rate, we estimate the state of mental health. The system according to feature 1.

7. The aforementioned counseling guidance unit is Based on the results analyzed by the aforementioned integrated analysis unit, the system will guide the user to counseling as needed. The system according to feature 1.

8. The aforementioned facial recognition unit is The system estimates the subject's emotions and then analyzes changes in facial color in more detail based on those estimated emotions. The system according to feature 1.

Images