Mental disorder analysis device

Abstract

To provide a device for analyzing whether a person has a mental disorder. [Solution] The system of this disclosure includes a storage unit that stores a mental disorder learning model that has been trained using training videos including patients diagnosed with mental disorders as training data; an acquisition unit that acquires analysis target videos including the person to be analyzed; and an analysis unit that applies the mental disorder learning model to the analysis target videos to analyze whether or not the person to be analyzed has the mental disorder.

Description

Technical Field

[0001] The present invention relates to a mental disorder analysis device.

Background Art

[0002] Techniques for analyzing the emotions that others feel towards the speech of a speaker are known (see, for example, Patent Document 1). Also known are techniques for analyzing the changes in the facial expressions of a subject over a long period of time in a time series and estimating the emotions held during that period (see, for example, Patent Document 2). Furthermore, techniques for identifying the factors that most influenced the change in emotions are known (see, for example, Patent Documents 3 to 5). Still further, techniques for comparing the normal facial expression of a subject with the current facial expression and issuing an alert when the facial expression is gloomy are known (see, for example, Patent Document 6). Also known are techniques for comparing the facial expression of a subject in a normal state (when expressionless) with the current facial expression and determining the degree of the subject's emotion (see, for example, Patent Documents 7 to 9). Furthermore, techniques for analyzing the emotions of an organization and the atmosphere within a group felt by an individual are also known (see, for example, Patent Documents 10 and 11).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Patent Document 5

Patent Document 6

Patent Document 7

Patent Document 8

[0004] While all the techniques described above allow for a multifaceted analysis of a subject, they cannot estimate their specific condition. Improvements are particularly needed to determine whether or not they are in a disease-related state.

[0005] The present invention aims to make it possible to determine whether or not a subject being analyzed is in a state related to a mental disorder. [Means for solving the problem]

[0006] According to the present invention A memory unit that stores a mental disorder learning model that has been trained using training videos, including those of patients diagnosed with mental disorders, as training data, An acquisition unit that acquires videos of the subjects to be analyzed, including the subjects of analysis, The system includes an analysis unit that applies the mental disorder learning model to the video to be analyzed to determine whether or not the person being analyzed has the mental disorder. A mental disorder analysis device can be obtained. [Effects of the Invention]

[0007] According to this disclosure, the burden of interviews in determining mental disorders can be reduced. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an overall diagram of the system according to an embodiment of the present invention. [Figure 2]An example of a functional block diagram according to an embodiment of the present invention. [Figure 3] A diagram showing a functional configuration example of a video analysis apparatus according to an embodiment of the present invention. [Figure 4] A diagram showing a functional configuration example of a mental disorder analysis apparatus according to an embodiment of the present invention. [Figure 5] A diagram showing the flow of a mental disorder apparatus. [Figure 6] A diagram showing the flow of a mental disorder apparatus.

Embodiments for Carrying Out the Invention

[0009] The contents of the embodiments of the present disclosure will be listed and described. The present disclosure has the following configuration. [Item 1] A storage unit that stores a mental disorder learning model obtained by learning a learning video including patients diagnosed with a mental disorder as teacher data, An acquisition unit that acquires an analysis target video including an analysis target person, An analysis unit that applies the mental disorder learning model to the analysis target video to analyze whether the analysis target person has the mental disorder. A mental disorder analysis apparatus. [Item 2] The mental disorder analysis apparatus according to claim 1, The learning video is a recording of the states of both a first subject and a second subject when they are undergoing the diagnosis process of the mental disorder by an expert, The first subject is a person who has been previously diagnosed with the mental disorder, and the second subject is a person who has not been previously diagnosed with the mental disorder. A mental disorder analysis apparatus. [Item 3] The mental disorder analysis apparatus according to claim 1, The analysis target video includes at least the expression and voice of the analysis target person. A mental disorder analysis apparatus. [Item 4] The mental disorder analysis apparatus according to claim 1, The video being analyzed is one that concerns a conversation between the subject and another person. A device for analyzing mental disorders. [Item 5] A mental disorder analysis apparatus according to claim 1, The aforementioned videos were obtained using online sessions. A device for analyzing mental disorders. [Item 6] A mental disorder analysis apparatus according to claim 1, The analysis unit includes information on whether the degree of the mental disorder of the person being analyzed exceeds a predetermined threshold. A device for analyzing mental disorders. [Item 7] A mental disorder analysis apparatus according to claim 1, A questioning unit that asks the aforementioned subjects of analysis a predetermined additional question, The system further includes an answer acquisition unit that obtains answers to the aforementioned additional questions, The analysis unit analyzes the responses to determine whether the subject of analysis, Korenam, has the aforementioned specific mental disorder. A device for analyzing mental disorders. [Item 8] The steps include storing a mental disorder learning model, which has been trained using training videos including patients diagnosed with mental disorders as training data, in the computer's memory, and Steps to obtain the video to be analyzed, including the person to be analyzed, The process includes the step of applying the mental disorder learning model to the video to be analyzed to analyze whether or not the person being analyzed has the mental disorder. Methods for analyzing mental disorders. [Item 9] A memory unit that stores a mental disorder learning model that has been trained using training videos, including those of patients diagnosed with mental disorders, as training data, An acquisition unit that acquires videos of the subjects to be analyzed, including the subjects of analysis, The system includes an analysis unit that applies the mental disorder learning model to the video to be analyzed to determine whether or not the person being analyzed has the mental disorder. Mental disorder analysis system. [Item 10] Computer A memory unit that stores a mental disorder learning model that has been trained using training videos, including those of patients diagnosed with mental disorders, as training data. Acquisition unit that acquires videos of the subjects to be analyzed, including the subjects of analysis. The aforementioned video subject to analysis is used as an analysis unit to analyze whether or not the subject of the analysis has the aforementioned mental disorder. Mental disorder analysis program.

[0010] Preferred embodiments of this disclosure will be described in detail below with reference to the attached drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0011] The present invention relates to an analysis system that analyzes video recordings of a subject communicating and performs an analysis to determine whether or not the subject is in a specific mental state (for example, mental disorders, but not limited to these).

[0012] This system generates a mental disorder learning model by training it with training videos containing patients diagnosed with mental disorders as training data. It then applies this mental disorder learning model to videos containing the target individual to analyze whether or not the target individual has a mental disorder. The generation of the mental disorder learning model and the analysis of the target individual are performed using a video analysis device described later.

[0013] The following describes the embodiments of this system in the following order. 1. Video analysis device Basic functions of a video analysis device Hardware configuration example How to obtain videos Analysis Flow 2. Model generation overview Model generation methods Other model generation methods 3. Analysis of the subjects of analysis Acquiring videos Application of the analytical model Obtaining numerical values Setting and determining thresholds Output format Examples of subjects of analysis

[0014] 1. Video analysis device <Basic Functions> The video analysis device of this embodiment is a system that analyzes and evaluates the unique emotions (feelings that arise in response to one's own or others' words and actions, such as pleasure, displeasure, or their degree) of a target individual among multiple people in an environment where a video session (hereinafter referred to as an online session, including both one-way and two-way) is conducted. Online sessions include, for example, online meetings, online classes, and online chats, where terminals installed in multiple locations are connected to a server via a communication network such as the internet, and video and images can be exchanged between multiple terminals through the server. The video and images handled in an online session include facial images and audio of users using the terminals. The video and images also include images such as materials that are shared and viewed by multiple users. It is possible to switch between displaying facial images and material images on the screen of each terminal to display only one of them, or to display facial images and material images simultaneously by dividing the display area. It is also possible to display the image of one of the multiple people in full screen, or to display the images of some or all users in a divided screen. It is possible to designate one or more of the multiple users participating in the online session using terminals as the target of analysis. For example, the leader, facilitator, or administrator of an online session (hereinafter collectively referred to as the organizer) can designate one of the users as the target of analysis. The organizer of an online session may be, for example, an instructor for an online class, a chair or facilitator for an online meeting, or a coach for a coaching session. The organizer of an online session is usually one of several users participating in the online session, but it may also be a different person who does not participate in the online session. Alternatively, all participants may be included in the analysis without designating a target. It is also possible for the leader, facilitator, or administrator of an online session (hereinafter collectively referred to as the organizer) to designate one of the users as the target of analysis. The organizer of an online session may be, for example, an instructor for an online class, a chair or facilitator for an online meeting, or a coach for a coaching session.The organizer of an online session is usually one of the multiple users participating in the session, but it may also be someone else who does not participate in the session.

[0015] The video analysis device according to this embodiment displays at least moving images acquired from a video session when a video session is established between multiple terminals. The displayed moving images are acquired by the terminals, and at least facial images contained within the moving images are identified at predetermined frame units. Subsequently, evaluation values ​​are calculated for the identified facial images. These evaluation values ​​are shared as needed. In particular, in this embodiment, the acquired moving images are stored on the terminal, analyzed and evaluated on the terminal, and the results are provided to the user of the terminal. Therefore, even video sessions containing personal information or confidential information can be analyzed and evaluated without providing the video itself to an external evaluation organization. Furthermore, if necessary, only the evaluation results (evaluation values) can be provided to external terminals to visualize the results or perform cross-analysis.

[0016] As shown in Figure 1, the video analysis device according to this embodiment includes user terminals 10 and 20 having at least an input unit such as a camera unit and a microphone unit, a display unit such as a display and an output unit such as a speaker, a video session service terminal 30 that provides a bidirectional video session to the user terminals 10 and 20, and an evaluation terminal 40 that performs part of the evaluation of the video session.

[0017] <Example Hardware Configuration> Each functional block, functional unit, and functional module described below can be configured using, for example, hardware, a DSP (Digital Signal Processor), or software provided in a computer. For example, when configured using software, it is actually configured using a computer's CPU, RAM, ROM, etc., and is realized by the operation of a program stored on a recording medium such as RAM, ROM, hard disk, or semiconductor memory. The series of processes performed by the system and terminal described herein may be realized using software, hardware, or a combination of software and hardware. It is possible to create a computer program to realize each function of the information sharing support device 10 according to this embodiment and implement it on a PC or the like. Furthermore, it is also possible to provide a computer-readable recording medium on which such a computer program is stored. Examples of recording media include magnetic disks, optical disks, magneto-optical disks, and flash memory. The above-mentioned computer program may also be distributed, for example, via a network, without using a recording medium.

[0018] The evaluation terminal 40 in this embodiment acquires video footage from a video session service terminal, identifies at least face images contained within the video footage at predetermined frame units, and calculates evaluation values ​​for the face images (details will be described later).

[0019] <How to obtain the video> As shown in Figure 2, the video session service provided by the video session service terminal (hereinafter sometimes simply referred to as "this service") enables bidirectional communication of images and audio to user terminals 10 and 20. This service displays video images acquired by the camera unit of the other user terminal on the display of the user terminal, and outputs audio acquired by the microphone unit of the other user terminal through the speaker. Furthermore, this service is configured to allow either or both user terminals to record video images and audio (collectively referred to as "video images, etc.") in the storage of at least one of the user terminals. The recorded video information Vs (hereinafter referred to as "recorded information") is cached on the user terminal that initiated the recording and stored only locally on one of the user terminals. Users can, if necessary, view the recorded information themselves, share it with others, etc., within the scope of using this service.

[0020] <Analysis Flow> Figure 3 is a block diagram showing an example configuration according to this embodiment. As shown in Figure 3, the video analysis device of this embodiment is realized as a functional configuration of a user terminal 10. Specifically, the user terminal 10 includes, as its functions, a video image acquisition unit 11, a biological reaction analysis unit 12, a specific determination unit 13, a related event identification unit 14, a clustering unit 15, and an analysis result notification unit 16.

[0021] The video acquisition unit 11 acquires video from each terminal by capturing multiple people (multiple users) using the cameras installed in each terminal during an online session. The video acquired from each terminal does not depend on whether or not the video is set to be displayed on the screen of that terminal. In other words, the video acquisition unit 11 acquires video from each terminal, including both video currently displayed and video currently hidden on the terminal.

[0022] The biological response analysis unit 12 analyzes changes in biological responses for each of several people based on the moving images (regardless of whether they are displayed on the screen or not) acquired by the moving image acquisition unit 11. In this embodiment, the biological response analysis unit 12 separates the moving images acquired by the moving image acquisition unit 11 into a set of images (a collection of frame images) and sound, and analyzes changes in biological responses from each.

[0023] For example, the bioresponse analysis unit 12 analyzes the user's face image using frame images separated from the video acquired by the video acquisition unit 11 to analyze changes in bioresponses related to at least one of the following: facial expression, gaze, pulse rate, and facial movement. In addition, the bioresponse analysis unit 12 analyzes changes in bioresponses related to at least one of the user's speech content and voice quality by analyzing the audio separated from the video acquired by the video acquisition unit 11.

[0024] When a person's emotions change, it manifests as changes in biological responses such as facial expressions, eye contact, pulse rate, facial movements, speech content, and voice quality. In this embodiment, changes in the user's emotions are analyzed by analyzing changes in the user's biological responses. One example of the emotion analyzed in this embodiment is the degree of pleasure / displeasure. In this embodiment, the biological response analysis unit 12 calculates a biological response index value that reflects the content of the changes in biological responses by quantifying the changes in biological responses according to predetermined criteria.

[0025] The analysis of facial expression changes is performed, for example, as follows: For each frame image, the facial region is identified within the frame image, and the identified facial expressions are classified into several categories according to a pre-trained image analysis model. Based on the classification results, the system analyzes whether positive or negative facial expression changes have occurred between consecutive frame images, and the magnitude of these changes, and outputs an facial expression change index value corresponding to the analysis results.

[0026] The analysis of changes in eye movement is performed, for example, as follows: For each frame image, the eye region is identified within the frame image, and the direction of both eyes is analyzed to determine where the user is looking. For example, it is analyzed whether the user is looking at the speaker's face, the shared document being displayed, or looking off-screen. It may also be possible to analyze whether the eye movement is large or small, and whether the movement is frequent or infrequent. Changes in eye movement are also related to the user's level of concentration. The bio-response analysis unit 12 outputs an eye movement change index value according to the analysis results of the changes in eye movement.

[0027] The analysis of pulse rate changes is performed, for example, as follows: For each frame image, the facial region is identified within the frame image. Then, the change in the G color of the facial surface is analyzed using a pre-trained image analysis model that captures the numerical value of the facial color information (G in RGB). By arranging the results along the time axis, a waveform representing the change in color information is formed, and the pulse rate is identified from this waveform. A person's pulse rate increases when they are nervous and decreases when they are calm. The biological response analysis unit 12 outputs a pulse rate change index value according to the analysis results of the pulse rate change.

[0028] The analysis of changes in facial movement is performed, for example, as follows: For each frame image, the facial region is identified within the frame image, and the orientation of the face is analyzed to determine where the user is looking. For example, it is analyzed whether the user is looking at the face of the speaker currently displayed, the shared document currently displayed, or looking off-screen. It may also be analyzed whether the facial movement is large or small, and whether the movement is frequent or infrequent. It may also be analyzed in conjunction with eye movement. For example, it may be analyzed whether the user is looking straight at the face of the speaker currently displayed, looking upwards or downwards, or looking at it from an angle. The bio-response analysis unit 12 outputs a facial orientation change index value according to the analysis results of the changes in facial orientation.

[0029] The analysis of the spoken content is performed, for example, as follows: The bioreaction analysis unit 12 converts the speech into a string by performing known speech recognition processing on the speech for a specified time (for example, a time of about 30 to 150 seconds), and removes unnecessary words that represent the conversation, such as particles and articles, by performing morphological analysis on the string. Then, it vectorizes the remaining words and analyzes whether a positive or negative emotional change has occurred, and to what extent the emotional change has occurred, and outputs a spoken content index value according to the analysis result.

[0030] Voice quality analysis is performed, for example, as follows: The bioreaction analysis unit 12 identifies the acoustic characteristics of the speech by performing known speech analysis processing on the speech for a specified time (for example, a time of about 30 to 150 seconds). Based on these acoustic characteristics, it analyzes whether a positive or negative voice quality change has occurred, and to what extent the voice quality change has occurred, and outputs a voice quality change index value according to the analysis results.

[0031] The bioresponse analysis unit 12 calculates a bioresponse index value using at least one of the facial expression change index value, eye gaze change index value, pulse rate change index value, face direction change index value, speech content index value, and voice quality change index value calculated as described above. For example, the bioresponse index value is calculated by weighting the facial expression change index value, eye gaze change index value, pulse rate change index value, face direction change index value, speech content index value, and voice quality change index value.

[0032] The uniqueness determination unit 13 determines whether the changes in biological responses analyzed for the subject of analysis are specific to those analyzed for other individuals. In this embodiment, the uniqueness determination unit 13 determines whether the changes in biological responses analyzed for the subject of analysis are specific to those analyzed for other individuals, based on the biological response index values ​​calculated for each of the multiple users by the biological response analysis unit 12.

[0033] For example, the uniqueness determination unit 13 calculates the variance of the bioresponse index values ​​calculated for each of the multiple individuals by the bioresponse analysis unit 12, and by comparing the bioresponse index value calculated for the subject of analysis with the variance, it determines whether the changes in the bioresponse analyzed for the subject of analysis are unique compared to others.

[0034] There are three possible patterns in which the changes in biological responses analyzed in the subject may be specific compared to others. The first is when no particularly large changes in biological responses occur in others, but relatively large changes occur in the subject. The second is when no particularly large changes in biological responses occur in the subject, but relatively large changes occur in others. The third is when relatively large changes in biological responses occur in both the subject and others, but the nature of the changes differs between the subject and others.

[0035] The related event identification unit 14 identifies events that occur with respect to at least one of the subject of analysis, other people, and the environment when a change in biological response determined to be specific by the specificity determination unit 13 occurs. For example, the related event identification unit 14 identifies the subject's own words and actions from the video when a specific change in biological response occurs for the subject of analysis. The related event identification unit 14 also identifies the words and actions of other people from the video when a specific change in biological response occurs for the subject of analysis. Furthermore, the related event identification unit 14 identifies the environment from the video when a specific change in biological response occurs for the subject of analysis. The environment may include, for example, shared documents displayed on the screen or objects visible in the background of the subject of analysis.

[0036] The clustering unit 15 analyzes the degree of correlation between changes in biological responses determined to be specific by the specificity determination unit 13 (for example, one or more combinations of eye gaze, pulse rate, facial movements, speech content, and voice quality) and the events that occur when such specific changes in biological responses occur (events identified by the related event identification unit 14). If it is determined that the correlation is above a certain level, the unit clusters the subjects of analysis or events based on the results of the correlation analysis.

[0037] For example, if a specific change in biological response corresponds to a negative emotional change, and the event occurring when that specific change in biological response occurs is also a negative event, a correlation of a certain level or higher will be detected. The clustering unit 15 clusters the subjects of analysis or events into one of several pre-segmented classifications according to the content of the event, its degree of negativity, the magnitude of the correlation, etc.

[0038] Similarly, if a specific change in biological response corresponds to a positive change in emotion, and the event occurring when that specific change in biological response occurs is also a positive event, a correlation of a certain level or higher will be detected. The clustering unit 15 clusters the subjects of analysis or events into one of several pre-segmented classifications according to the content of the event, its degree of positivity, the magnitude of the correlation, etc.

[0039] The analysis result notification unit 16 notifies the person who designated the analysis subject (the analysis subject or the organizer of the online session) of at least one of the changes in biological responses determined to be specific by the anomaly determination unit 13, the events identified by the related event identification unit 14, and the classifications clustered by the clustering unit 15.

[0040] For example, the analysis result notification unit 16 notifies the analysis subject of their own words and actions as events occurring when a specific change in biological response occurs in the analysis subject that differs from that of others (one of the three patterns described above; the same applies hereinafter). This allows the analysis subject to understand that they have different emotions than others when they perform certain words and actions. At this time, the analysis subject may also be notified of the specific change in biological response identified for the analysis subject. Furthermore, the analysis subject may also be notified of the change in biological response of the other person being compared.

[0041] For example, if there is a discrepancy between the emotions others felt when an individual, in their usual, unconscious actions, or when they consciously performed actions accompanied by a particular emotion, is perceived by others, the individual will be notified of their own actions at that time. This makes it possible to discover actions that are well-received or poorly received by others, contrary to one's own awareness.

[0042] Furthermore, the analysis result notification unit 16 notifies the online session organizer of the events occurring when a unique change in the biological response of the person being analyzed occurs, along with the change in the unique biological response. This allows the online session organizer to understand what kinds of events are influencing what kinds of emotional changes as phenomena unique to the designated person being analyzed. Based on this understanding, it becomes possible to take appropriate measures for the person being analyzed.

[0043] Furthermore, the analysis result notification unit 16 notifies the online session organizer of the event or the clustering result of the analyzed subject when a unique change in biological response occurs in the analyzed subject that differs from that of others. This allows the online session organizer to understand the behavioral tendencies unique to the analyzed subject, predict future behaviors and conditions, and take appropriate action for the analyzed subject based on which classification the specified analyzed subject has been clustered into.

[0044] 2. Model generation <Overview> The system according to this embodiment includes a storage unit that stores a mental disorder learning model trained on training videos containing patients diagnosed with mental disorders as training data, an acquisition unit that acquires videos containing the subject of analysis, and an analysis unit that applies the mental disorder learning model to the videos to analyze whether or not the subject of analysis has a mental disorder.

[0045] <Model generation methods> Mental disorder learning models use videos as training data that capture the characteristics of patients diagnosed with mental disorders, such as their behavior, facial expressions, and speech. This allows the model to learn the characteristics and patterns of mental disorders, enabling it to analyze new video data for the presence or absence of mental disorders.

[0046] In particular, the learning video according to this embodiment is a recording of both the first and second subjects undergoing the diagnostic process for the mental disorder by a specialist. Here, the first subject is a person who has been diagnosed with a mental disorder in advance, and the second subject is a person who has not been diagnosed with a mental disorder in advance.

[0047] The first subject is someone who has already been diagnosed with a mental disorder before the video was filmed. This subject's video serves as primary training data for the model to learn the distinct characteristics of the mental disorder. From the first subject's video, the model can learn typical symptoms, reactions, facial expressions, speech patterns, and other characteristics of the mental disorder.

[0048] The second subject is someone who was not diagnosed with a mental disorder before the video was filmed. This subject's video helps the model learn characteristics of normal responses and behaviors. It is also essential for preventing overfitting and for improving the model's ability to learn characteristics of normal behavior and determine the absence of a mental disorder.

[0049] As shown in Figure 5, the specific procedure for generating a mental disorder learning model is as follows:

[0050] <Step 1> The first and second subjects are asked to converse with an expert, and the conversation is recorded. Both video (visual information) and audio (auditory information) data are obtained.

[0051] As an example of the diagnostic process described above, the "diagnostic criteria for depression" are as follows: That is, the diagnosis of depression is based on the patient's self-reported symptoms, clinical observations, and standard diagnostic criteria (e.g., DSM-5 or ICD-10). For example, a specialist will conduct a medical interview to determine if the following symptoms are present: 1. Low mood or depressed mood 2. Loss of interest or pleasure 3. Fatigue and loss of energy 4. Feelings of worthlessness or excessive or inappropriate guilt 5. Decreased concentration and difficulty making decisions 6. Insomnia or hypersomnia 7. Weight changes associated with increased or decreased appetite 8. Thoughts or plans about self-harm or suicide

[0052] <Step 2> More specifically, conversations between experts and subjects will be conducted according to the diagnostic process for clinically recognized mental disorders. This process will clearly capture the signs and characteristics of mental disorders, such as the subject's reactions, facial expressions, word choice, and tone, contributing to model generation. Through conversations with patients, experts may assess the presence and severity of the aforementioned depressive symptoms, or label the subject. Information on the patient's daily life and activity levels, past medical history, family medical history, medications and other treatments being used may also be collected.

[0053] Examples of conversations with experts that may take place include the following: 1. Expert: "How have you been feeling lately? How does your mood change from day to day?" Subject: "I've been depressed all the time, and I don't feel like doing anything. I feel like I can't enjoy anything." 2. Expert: "How is your sleep at night? Do you wake up multiple times during the night?" Subject: "I wake up many times during the night, and in the morning I'm too tired to get up." 3. Expert: "Have you noticed any changes in your eating habits recently? Has your appetite increased or decreased?" Subject: "I have almost no appetite, and even eating feels like a chore." 4. Expert: "How do you feel about yourself? Have your perceptions of self-esteem or self-worth changed?" Subject: "I often feel like I can't do anything, and my sense of worthlessness is getting stronger."

[0054] <Step 3> The conversations of all subjects are analyzed using multimodal AI to extract features and generate a learning model. By utilizing multimodal AI, information obtained from both video and audio can be maximized to generate a highly accurate learning model for mental disorders.

[0055] <Other Model Generation Methods> The model according to this embodiment may be generated by, for example, the following method. Supervised learning: This method uses videos of patients diagnosed with mental disorders combined with their diagnostic results (labels). The model learns by extracting features from the videos and associating them with the labels. Transfer learning: This method uses existing video recognition or face recognition models as a foundation to train models on features specific to mental disorders. A model pre-trained on a large amount of general video data is used, and then fine-tuned with a small amount of data on mental disorders. Deep Learning: This method uses neural networks to capture deep features of image and audio data within videos. CNNs (Convolutional Neural Networks) learn visual features within videos, while RNNs (Recurrent Neural Networks) and LSTMs (Long Short-Term Memory) learn features with temporal continuity. Reinforcement learning: Based on diagnostic results, the model learns by rewarding it when it makes a correct judgment, allowing it to learn repeatedly. This helps the model learn the actions that produce the most accurate analysis results. By appropriately combining these learning methods, a learning model capable of accurately capturing the characteristics of mental disorders can be generated.

[0056] 3. Analysis of the subjects of analysis <Getting the video> Record the subjects of the analysis while they are having a normal conversation.

[0057] In this embodiment, everyday conversation refers to linguistic communication as social interaction that occurs naturally among people present, rather than being conducted for a specific job or purpose. For example, conversations in a medical setting can include professional conversations aimed at examination and diagnosis (conversations conducted as part of medical procedures, such as confirming symptoms, explaining test results, and discussing treatment plans) and everyday conversations (natural dialogues unrelated to medical treatment, such as small talk in the waiting room, talking about the weather before and after an examination, or chatting about family matters). Everyday conversations are characterized by their low purpose (not primarily aimed at solving a specific problem or gathering information) and their spontaneous nature (not being planned). In other words, even within the same medical setting, conversations for interviews and diagnoses are excluded from "everyday conversations," while small talk exchanged between examinations is included. It may also be required that everyday conversations be bidirectional (not one-sided information transmission, but mutual exchange), but in this embodiment, situations where the patient is speaking unilaterally while the doctor is listening are also included in everyday conversations. Furthermore, everyday conversations may also be recognized as having a function of maintaining and building social relationships.

[0058] Recording everyday conversations could include recordings of daily online meetings, for example. It could also be used to simulate conversations with friends and family in daily life, or interactions with computer-generated agents. In the case of agents, they could be programmed to ask general everyday questions or questions designed to elicit deeper emotions, and the responses of the person being analyzed could be observed.

[0059] <Recording on the device of the person being analyzed / partner> Recording of everyday conversations can be performed, for example, by a user terminal 10 operated by the person being analyzed. Alternatively, recording of everyday conversations may be performed by a user terminal 10 operated by a partner, such as a family member (parent, child, sibling, etc.) or friend of the person being analyzed. The user terminal 10 may be, for example, a smartphone, smartwatch, or personal computer, and is assumed to be equipped with a camera and a microphone. The user terminal 10 can, for example, capture video calls and acquire video (which may include audio) in the background. The application used to perform the video call may be, for example, a telephone application, a video conferencing application, or a chat application. It may be any application installed on the user terminal 10, either all or some of them. Furthermore, if an online session is conducted via a video session service terminal 30, the system may acquire the video data accumulated by the video session service terminal 30. While the person being analyzed is making an audio call with the device held away from their ear, the inline camera of the mobile device may capture still images or videos of the person being analyzed during the call. The user terminal 10 can capture these video calls continuously, periodically, at random intervals, or at any pre-configured interval. The user terminal 10 can transmit the acquired videos to the evaluation terminal 40. Alternatively, the user terminal 10 (the user terminal 10 of the person being analyzed and / or the partner's user terminal 0) may be equipped with a learning model, and the user terminal 10 may perform inferences about the presence or absence of mental disorders. In this case, the videos can be stored in the user terminal 10 without being transmitted to the evaluation terminal 40.

[0060] <Recording the conversation with the agent> Recording of everyday conversations can be performed on a terminal providing an agent (for example, a video session service terminal 30). The video session service terminal 30 can display an avatar as an agent, generate audio data of the conversation, and play the audio data back to the person being analyzed. The audio data of the conversation can be generated, for example, by providing a large-scale language model with prompts that include text data obtained from the content of the person being analyzed using speech recognition, etc., and instructions to create conversation content in response to this text. The video session service terminal 30 can acquire video data of the conversation between the person being analyzed and the agent (which may include audio data of the utterances and video data of the person being analyzed, for example, captured by an inline camera).

[0061] <Recording at medical facilities> When a doctor interviews a patient (analysis subject) (at least before, during, or after the interview), small talk between the doctor and the patient, or between a medical professional such as a nurse and the patient, may be recorded and acquired as a video of everyday conversation. For example, a user terminal 10 operated by a medical professional can record a scene of a patient and a medical professional having an everyday conversation and transmit it to an evaluation terminal 40. The user terminal 10 operated by the medical professional may be, for example, a medical terminal for viewing medical record information, or it may be a terminal other than a medical terminal. If the medical professional's user terminal 10 records a scene of an analysis subject's everyday conversation, the medical professional's user terminal 10 may transmit the video data to the evaluation terminal 40, or the medical professional's user terminal 10 may store a learning model and perform inferences about the presence or absence of mental disorder based on the video data.

[0062] <Application of the analytical model> The acquired video is input into a multimodal AI. The multimodal AI can be a learning model trained using machine learning on video data including patients diagnosed with mental disorders. Alternatively, the multimodal AI can be a learning model created using machine learning, with training data including, for example, the results of voice quality analysis, the results of text analysis obtained by converting speech to text using speech recognition, the results of detecting biological responses such as emotions through video analysis, and the presence or absence of mental disorders. For the analysis, information is obtained from both video data (facial expressions, body language, etc.) and audio data (speaking style, word choice, etc.). Using the previously generated mental disorder learning model, it is possible to analyze the features and patterns contained in videos of the subject's daily conversations and determine whether there are signs or characteristics of mental disorder.

[0063] <Getting numerical values> The system may include an output unit that outputs the results of the AI ​​analysis as numerical data. This numerical value may be, for example, between 0 and 1, with a value closer to 1 indicating a stronger presence of mental disorder characteristics. This numerical value is used as an indicator of the probability or degree of mental disorder. The output unit may be provided on the evaluation terminal 40 or on the user terminal 10.

[0064] <Setting and determining thresholds> Based on pre-set thresholds, the system determines whether an individual may have a mental disorder. For example, a score of 0.7 or higher might indicate a high probability of showing signs of a mental disorder. These thresholds are set based on previous research and clinical experience and may be updated as research progresses. Ultimately, this system can enable the early detection of mental disorders.

[0065] <Output format> The aforementioned assessment results will be output in a predetermined format. Examples of output formats include PDF reports, notifications via email, display on dashboards of HR cloud services, and API integration. Examples of recipients of the output include medical professionals such as attending physicians and counselors, the individual themselves, their family, management departments of organizations such as companies and schools, insurance companies, and rehabilitation facilities.

[0066] <Output during a medical interview with a doctor> When outputting the assessment results to a physician, the system may include a results display unit that displays the assessment results during the physician's patient interview. The results display unit can display the assessment results on the physician's terminal. This allows the physician to refer to the presence or absence of a mental disorder, or the possibility of a mental disorder, as a reference during diagnosis, based on everyday conversation.

[0067] The physician's terminal can obtain from the evaluation terminal 40 the presence or possibility of a mental disorder inferred using video data of everyday conversation transmitted from the user terminal 10 of the person being analyzed to the evaluation terminal 40. In addition, if the presence or absence of a mental disorder is inferred on the user terminal 10 of the person being analyzed, their partner, or a healthcare professional, the presence or possibility of a mental disorder may be obtained from the user terminal 10.

[0068] The system according to this embodiment may include a consent acquisition unit that obtains consent from the person being analyzed and / or a partner (related party) for the disclosure of the judgment results to the physician. The results display unit can display the judgment results on the physician's terminal when consent has been obtained from both the person being analyzed and the partner.

[0069] <Output to the analysis target> Output corresponding to the judgment result can be generated from the user terminal 10 (e.g., a smartphone) of the person being analyzed.

[0070] (A message expressing concern) For example, the output unit can output a message expressing concern to the person being analyzed, depending on the judgment result. A message expressing concern could be a question such as, "What's wrong?" The message from the output unit may be displayed as text on the user terminal 10 of the person being analyzed, or it may be played back as audio data generated by speech synthesis processing on the user terminal 10.

[0071] (Smart home control) For example, the output unit may include a control unit that controls devices placed in the analysis subject's residence according to the determination result. These devices could be, for example, lighting fixtures, music playback devices such as smart speakers, or display devices such as televisions or monitors.

[0072] The control unit can control the lighting fixtures to have different brightness levels depending on the judgment result. For example, if the person being analyzed is determined to have a mental disorder, or if the probability of having a mental disorder exceeds a predetermined value, the control unit can turn on the lighting fixtures or increase the brightness of the lighting fixtures.

[0073] The control unit can, for example, control a smart speaker or a music player on a personal computer to play different songs depending on the judgment result. For example, if the control unit determines that the person being analyzed has a mental disorder, or if the probability of having a mental disorder is above a predetermined value, it can select and play an up-tempo song (for example, a song whose BPM (Beats Per Minute) is greater than a predetermined threshold).

[0074] (Displaying the photo) The control unit can control the display of different photographs according to the judgment result. The photograph data is stored, for example, in a predetermined storage unit (photo data storage unit). Furthermore, the display history of photographs viewed by the person being analyzed is stored in a predetermined storage unit (display history storage unit). The photograph data storage unit and the display history storage unit may be provided, for example, in the user terminal 10, or in the evaluation terminal 40, allowing the user terminal 10 to access the photograph data. The control unit can display different photographs on a display device (for example, a mobile device such as a smartphone, or a display device such as a monitor or smart TV) according to the judgment result. In this case, the control unit can select the photograph to display according to the display history of the photograph data. For example, if the control unit is determined to have a mental disorder, or if the probability of having a mental disorder is above a predetermined value, it can display photographs that the person being analyzed frequently views. Whether the person is frequently viewing the photographs can be determined by whether the number of views is above a predetermined number (for example, the average number of views or a fixed value). The number of views may be calculated by counting the number of views over a specific period (this could be the entire period, the most recent period, or a predetermined period in the past (for example, the past year, the year of viewing, etc.)).

[0075] Furthermore, if the user terminal 10 can capture images of the person being analyzed viewing photo data, the analysis unit can analyze the images (video or still images) to detect biological responses such as emotions. The control unit aggregates the number of times a given positive emotion was detected during viewing or the viewing time during which the user was viewing with a given positive emotion, and based on these aggregated values ​​(number of times, time, or a combination thereof), it can identify photo data that is expected to evoke positive emotions during viewing. The control unit can then control the display of the identified photo data that is expected to evoke positive emotions during viewing.

[0076] (Suggestion to make a call) The output unit can suggest calls to different call partners depending on the determination result. For example, the system may include a call history storage unit that stores the call history of the person being analyzed. The call history includes information that identifies multiple callers (at least two) and the date and time of the call. The output unit can search the call history that contains information that identifies the person being analyzed. From the searched call history, the output unit identifies the call partners of the person being analyzed and aggregates the number of calls and / or call duration for each call partner. The aggregation of the number of calls and / or call duration may be performed for specific periods (this may be the entire period, the most recent period, or a predetermined long period in the past (e.g., one year from the present, the year or fiscal year viewed, etc.)). If the person being analyzed is determined to have a mental disorder, or if the possibility of having a mental disorder is above a predetermined value, the output unit can identify call partners with whom the person being analyzed has called a predetermined number of times and / or for a predetermined duration and suggest calls to the identified call partners.

[0077] Furthermore, if the user terminal 10 can capture footage of the person being analyzed during a call (or if the call is conducted via an online session and video data can be obtained from the video session service terminal 30), the analysis unit can analyze the images (video or still images) to detect biological responses such as emotions. The output unit aggregates the number of times a given positive emotion was detected during the call or the duration of the call while the person was experiencing that positive emotion. Based on these aggregated values ​​(number of times, duration, or a combination thereof), the output unit can identify the call partner with whom positive emotions are expected during the call. The output unit can then suggest a call with the identified call partner with whom positive emotions are expected during the call.

[0078] Furthermore, by including the call content (which may be video data, audio data, or text data of the call content obtained through voice analysis) in the call history, the analysis unit can determine the stress level of the person being analyzed during the call as a biological response. The analysis unit aggregates the number of times the stress level exceeded a predetermined value during the call, and the duration for which the stress level exceeded the predetermined value, and based on these aggregated values ​​(number of times, duration, or a combination thereof), it can identify the caller who caused the person being analyzed stress during the call. The output unit can then suggest callers from among the callers of the person being analyzed, excluding those who caused the person being analyzed stress during the call that were identified.

[0079] (Action proposal) The output unit may include a decision unit that determines the action the subject of analysis should take based on the judgment result. The output unit can propose the action determined by the decision unit to the subject of analysis (for example, by displaying it as text or notifying them by voice). For example, if the user is having a conversation with an avatar, the action can be proposed as an utterance from the avatar.

[0080] The system may include an action memory unit that stores actions associated with candidate judgment results. The decision unit can retrieve the action corresponding to the judgment result from the action memory unit.

[0081] Furthermore, the decision unit can generate actions by providing a large-scale language model with a prompt that includes a judgment result and instructions for generating actions that the person who received the judgment should take. It can also select one or more actions by providing the large-scale language model with a prompt that includes multiple actions stored in the action memory unit, the judgment result, and instructions for selecting one or more actions from the multiple actions based on the judgment result. The output unit can propose the selected actions to the person being analyzed.

[0082] <Examples of subjects of analysis> The disorders targeted by this system include, for example, the following: depression, bipolar disorder, schizophrenia, general anxiety disorder, obsessive-compulsive disorder, panic disorder, social anxiety disorder, PTSD, borderline personality disorder, antisocial personality disorder, dependent personality disorder, avoidant personality disorder, attention-deficit / hyperactivity disorder, autism spectrum disorder, alcohol use disorder, drug use disorder, Alzheimer's disease, Lewy body dementia, vascular dementia, anorexia nervosa, bulimia nervosa, insomnia, sleep apnea syndrome, hypoactive libido, sexual dysfunction, conversion disorder, dissociative disorder, sumatform disorder, pseudo-disorder caused by others, and pseudo-disorder caused by oneself.

[0083] The processes described using flowcharts in this specification do not necessarily have to be executed in the order shown. Some processing steps may be executed in parallel. Additional processing steps may be adopted, and some processing steps may be omitted.

[0084] The embodiments described above may be combined as appropriate. Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that will be obvious to those skilled in the art from the description herein, in addition to or instead of the effects described above.

[0085] <Disclosure Items> Furthermore, the disclosures in this specification include the following:

[0086] Item 1: Display results from everyday conversation during the medical interview. [Item 1-1] (Assess the presence or absence of mental disorder from everyday conversation and display it on the doctor's terminal during the medical interview.) A memory unit that stores a mental disorder learning model that has been trained using training videos, including those of patients diagnosed with mental disorders, as training data, An acquisition unit that acquires videos related to the daily conversations of the subjects of analysis, A determination unit that provides the mental disorder learning model with the video to be analyzed and determines whether or not the person being analyzed has the mental disorder, During a medical interview with the subject of analysis, the results display unit on the terminal used by the physician displays the determination result of whether or not the person has a mental disorder. An information processing system characterized by comprising the following features.

[0087] Item 2: The partner (spouse, parent, etc.) of the person being analyzed determines whether or not the person being analyzed has a mental disorder based on conversations between the partner and the person being analyzed. [Item 2-1] (The partner of the person being analyzed understands their partner's mental state.) A memory unit that stores a mental disorder learning model that has been trained using training videos, including those of patients diagnosed with mental disorders, as training data, An acquisition unit acquires video data of a video call relating to a daily conversation between a first terminal of a person related to the person being analyzed and a second terminal of the person being analyzed, A determination unit that provides the mental disorder learning model with the video to be analyzed and determines whether or not the person being analyzed has the mental disorder, An information processing system characterized by comprising the following features. [Item 2-2] (Combination with Item 1) The information processing system described in item 2-1 of the tree, A storage unit that stores the determination result from the determination unit, During a medical interview with the subject of analysis, the results display unit on the terminal used by the physician displays the determination result of whether or not the person has a mental disorder. An information processing system characterized by comprising the following features. [Item 2-3] An information processing system as described in item 2-1 or item 2-2, A determination result storage unit that stores the determination result from the determination unit, A consent acquisition unit that obtains consent from the subject of analysis and related parties for the disclosure of the judgment results to the physician, Equipped with, The results display unit will display the judgment result on the terminal when consent is obtained from both the person being analyzed and the person concerned. An information processing system characterized by the following.

[0088] Item 3: Determine whether or not the subject has a mental disorder based on conversations conducted using the subject's mobile device (such as a smartphone). [Item 3-1] (Assess the mental state of the subject using their mobile device.) A memory unit that stores a mental disorder learning model that has been trained using training videos, including those of patients diagnosed with mental disorders, as training data, An acquisition unit acquires video data of a video call related to a daily conversation that the subject of analysis is having on the subject's terminal, and A determination unit that provides the mental disorder learning model with the video to be analyzed and determines whether or not the person being analyzed has the mental disorder, An information processing system characterized by comprising the following features. [Item 3-2] (In combination with Item 5, such as "What's wrong?") The information processing system described in item 3-1, The system includes an output unit that outputs a message expressing concern to the person being analyzed, in accordance with the determination result from the determination unit. An information processing system characterized by the following. [Item 3-3] (Combination with Item 5, Smart Home) An information processing system as described in item 3-1 or item 3-2, The system includes a control unit that controls a device placed in the residence of the person being analyzed, in accordance with the determination result from the determination unit. An information processing system characterized by the following. [Item 3-4] (Turn up the lights) The information processing system described in item 3-3, The control unit adjusts the brightness of the lighting fixture according to the determination result. An information processing system characterized by the following. [Item 3-5] (Play an upbeat song) An information processing system as described in item 3-3 or item 3-4, The control unit plays different music according to the determination result. An information processing system characterized by the following. [Items 3-6] (Combination with Item 5, display of photos) An information processing system described in any one of items 3-1 to 3-5, A photo storage unit that stores photo data, A display history storage unit that stores the display history of the aforementioned photographic data, A photo display unit that displays the photo that the person being analyzed has viewed at a predetermined frequency or more, based on the display history, in accordance with the determination result by the determination unit, An information processing system characterized by comprising the following features. [Items 3-7] (Combination with Item 5, recommendation to make a phone call) An information processing system described in any one of items 3-1 to 3-6, A call history storage unit that stores the call history of the person being analyzed, including the person the call was made to, A proposal unit proposes calls to the person the person being analyzed has spoken to more than a predetermined frequency, based on the call history, in accordance with the determination result by the determination unit. An information processing system characterized by comprising the following features. [Item 3-8] (Identifying stressors) The information processing system described in item 3-7, The aforementioned call history includes the content of the call. The system includes a stress determination unit that analyzes the content of the aforementioned phone call and determines the stress level of the person being analyzed. An information processing system characterized by the following. [Items 3-9] (Combinations with Item 1) An information processing system described in any one of items 3-1 to 3-8, During a medical interview with the subject of analysis, the terminal used by the physician is equipped with a result display unit that displays the determination result of whether or not the person has a mental disorder. An information processing system characterized by the following.

[0089] Item 4: Conversation with an AI avatar [Item 4-1] (Understanding the mental state of the subject through conversation with an AI avatar) A memory unit that stores a mental disorder learning model that has been trained using training videos, including those of patients diagnosed with mental disorders, as training data, A conversation processing unit that conducts everyday conversations via video call with the person being analyzed, An acquisition unit that acquires the video data to be analyzed, which is the video data related to the aforementioned video call, A determination unit that provides the mental disorder learning model with the video to be analyzed and determines whether or not the person being analyzed has the mental disorder, An information processing system characterized by comprising the following features. [Item 4-2] (Recommend the next action) The information processing system described in item 4-1, A decision unit determines the next action that the person being analyzed should take, in accordance with the determination result from the determination unit, A notification unit that notifies the subject of the analysis of the aforementioned action, An information processing system characterized by comprising the following features. [Item 4-3] (Have the LLM decide on the action) The information processing system described in item 4-2, The determination unit generates the action by providing a large-scale language model with a prompt that includes the determination result and instructions for generating the action that the person who received the determination result should take. An information processing system characterized by the following. [Item 4-4] (List the actions) The information processing system described in item 4-2, The system includes an action storage unit that stores the action in association with the candidate judgment result, The determination unit obtains the action corresponding to the determination result from the action storage unit. An information processing system characterized by the following. [Items 4-5] (List the actions and have the LLM select one) The information processing system described in item 4-3, It includes an action memory unit that stores multiple aforementioned actions, The decision unit causes the large-scale language model to select an action by providing a prompt that includes the plurality of actions, the determination result, and an instruction to select one or more of the plurality of actions based on the determination result. An information processing system characterized by the following. [Items 4-6] (Combination with Item 5, Smart Home) An information processing system described in any one of items 4-1 to 4-5, The system includes a control unit that controls a device placed in the residence of the person being analyzed, in accordance with the determination result from the determination unit. An information processing system characterized by the following. [Item 4-7] (Turn up the lights) The information processing system described in item 4-6, The control unit adjusts the brightness of the lighting fixture according to the determination result. An information processing system characterized by the following. [Item 4-8] (Play an upbeat song) An information processing system as described in item 4-6 or item 4-7, The control unit plays different music according to the determination result. An information processing system characterized by the following. [Items 4-9] (Combination with Item 5, display of photos) An information processing system described in any one of items 4-1 to 4-8, A photo storage unit that stores photo data, A display history storage unit that stores the display history of the aforementioned photographic data, A photo display unit that displays the photo that the person being analyzed has viewed at a predetermined frequency or more, based on the display history, in accordance with the determination result by the determination unit, An information processing system characterized by comprising the following features. [Items 4-10] (Combination with Item 5, recommendation to make a phone call) An information processing system described in any one of items 4-1 to 4-9, A call history storage unit that stores the call history of the person being analyzed, including the person the call was made to, A proposal unit proposes calls to the person the person being analyzed has spoken to more than a predetermined frequency, based on the call history, in accordance with the determination result by the determination unit. An information processing system characterized by comprising the following features. [Item 4-11] (Combination with Item 1) An information processing system described in any one of items 4-1 to 4-10, During a medical interview with the subject of analysis, the terminal used by the physician is equipped with a result display unit that displays the determination result of whether or not the person has a mental disorder. An information processing system characterized by the following.

[0090] Item 5: Light notification (Provide a mild notification when diagnosing a mental disorder) [Item 5-1] (Show concern, such as "What's wrong?") A memory unit that stores a mental disorder learning model that has been trained using training videos, including those of patients diagnosed with mental disorders, as training data, An acquisition unit that acquires videos related to the daily conversations of the subjects of analysis, A determination unit that provides the mental disorder learning model with the video to be analyzed and determines whether or not the person being analyzed has the mental disorder, The system includes an output unit that outputs a message expressing concern to the person being analyzed, in accordance with the determination result from the determination unit. An information processing system characterized by comprising the following features. [Item 5-2] (Smart Home) A memory unit that stores a mental disorder learning model that has been trained using training videos, including those of patients diagnosed with mental disorders, as training data, An acquisition unit that acquires videos related to the daily conversations of the subjects of analysis, A determination unit that provides the mental disorder learning model with the video to be analyzed and determines whether or not the person being analyzed has the mental disorder, The system includes a control unit that controls a device placed in the residence of the person being analyzed, in accordance with the determination result from the determination unit. An information processing system characterized by comprising the following features. [Item 5-3] (Turn up the lights) The information processing system described in item 5-2, The control unit adjusts the brightness of the lighting fixture according to the determination result. An information processing system characterized by the following. [Item 5-4] (Play an upbeat song) An information processing system as described in item 3-2 or item 5-3, The control unit plays different music according to the determination result. An information processing system characterized by the following. [Item 5-5] (Display of photo) An information processing system described in any one of items 5-1 to 5-4, A photo storage unit that stores photo data, A display history storage unit that stores the display history of the aforementioned photographic data, A photo display unit that displays the photo that the person being analyzed has viewed at a predetermined frequency or more, based on the display history, in accordance with the determination result by the determination unit, An information processing system characterized by comprising the following features. [Items 5-6] (Recommendation to make a phone call) An information processing system described in any one of items 5-1 to 5-5, A call history storage unit that stores the call history of the person being analyzed, including the person the call was made to, A proposal unit proposes calls to the person the person being analyzed has spoken to more than a predetermined frequency, based on the call history, in accordance with the determination result by the determination unit. An information processing system characterized by comprising the following features. [Item 5-7] (Identifying stressors) The information processing system described in item 5-6, The aforementioned call history includes the content of the call. The system includes a stress determination unit that analyzes the content of the aforementioned phone call and determines the stress level of the person being analyzed. An information processing system characterized by the following. [Explanation of symbols]

[0091] 10, 20 user terminals 30 Video Session Service Terminals 40 Evaluation terminals

Claims

1. A memory unit that stores a mental disorder learning model that has been trained using training videos, including those of patients diagnosed with mental disorders, as training data, An acquisition unit acquires video data of a video call relating to a daily conversation between a first terminal of a person related to the person being analyzed and a second terminal of the person being analyzed, A determination unit that provides the mental disorder learning model with the video to be analyzed and determines whether or not the person being analyzed has the mental disorder, An information processing system characterized by comprising the following features.

2. The information processing system according to claim 1, A storage unit that stores the determination result from the determination unit, During a medical interview with the subject of analysis, the results display unit on the terminal used by the physician displays the determination result of whether or not the person has a mental disorder. An information processing system characterized by comprising the following features.

3. The information processing system according to claim 2, A determination result storage unit that stores the determination result from the determination unit, A consent acquisition unit that obtains consent from the subject of analysis and related parties for the disclosure of the judgment results to the physician, Equipped with, The results display unit will display the judgment result on the terminal when consent is obtained from both the person being analyzed and the person concerned. An information processing system characterized by the following.

Images