Digital phenotyping-based depression analysis device and method

Abstract

The present invention relates to a digital phenotyping-based depression analysis device and method, the device comprising: a data collection unit for monitoring a digital learning process on a user terminal so as to collect multimodal data related to finger or stylus pen use-related behavior of a user; a data analysis unit for analyzing the multimodal data so as to detect at least one digital marker associated with a symptom of depression of the user; and a depression identification unit for determining the severity of the symptom of depression of the user and whether the user has depression in the digital learning process on the basis of the at least one digital marker.

Description

Digital phenotyping-based depression analysis device and method

[0001] The present invention relates to a technology for analyzing depression, and more specifically, to a digital phenotyping-based depression analysis device and method capable of screening a user's depressive symptoms and diagnosing and managing depression early by analyzing various digital indicators collected from potential behaviors through monitoring the user's use of a smart device.

[0002] Digital phenotyping refers to the process of collecting and quantifying a user's cognitive, emotional, behavioral, physiological, social, and environmental indicators at every moment of daily life using digital devices such as smartphones. Digital phenotyping is attracting attention as a new mental health measurement tool because it can provide multidimensional and large amounts of objective information. In the future, it is expected to be used as a tool for personalized medicine, preventive medicine, predictive medicine, participatory medicine, and precision medicine, which aim to improve the diagnostic accuracy of psychiatry by establishing a new dimensional diagnostic system for RDoC or reflecting individual patient data.

[0003] Recently, there have been attempts to collect and analyze users' biometric indicators, mobile phone usage patterns, and behavioral characteristics through mobile devices and wearable devices, but there are limitations in that the scope of application is very restricted in the collection and analysis of vast amounts of data.

[0004] In particular, public education settings such as schools are highly sensitive to personal information, and significant restrictions may exist on data collection due to factors like parental opposition. Therefore, there is a demand for digital phenotyping technology that can collect and utilize information—such as touch, stroke, and stylus pen usage behaviors—in a non-intrusive manner, excluding sensitive personal information like faces, facial expressions, body language, words and sentence structures in writing or speech, and social media posts.

[0005] [Prior Art Literature]

[0006] [Patent Literature]

[0007] Korean Publication No. 10-2021-0076462 (June 24, 2021)

[0008] One embodiment of the present invention aims to provide a digital phenotyping-based depression analysis device and method capable of screening depressive symptoms in a user and diagnosing and managing depression early by monitoring the user's use of a smart device and analyzing various digital indicators collected from potential behaviors.

[0009] Among the embodiments, a digital phenotyping-based depression analysis device comprises: a data collection unit that monitors a digital learning process on a user terminal to collect multimodal data regarding behaviors related to the use of a user's finger or stylus pen; a data analysis unit that analyzes the multimodal data to detect at least one digital indicator associated with the user's depressive symptoms; and a depression identification unit that determines the degree of the user's depressive symptoms and whether or not there is depression during the digital learning process based on the at least one digital indicator.

[0010] The data collection unit collects the multimodal data through an application executed on the user terminal during the digital learning process, and the multimodal data may include operation indicators based on the user's behavior on the display screen of the user terminal.

[0011] The above data collection unit can collect the user's keyboard or stylus pen-related actions and stroke actions collected by the on-screen keyboard sensor and touchscreen sensor of the user terminal during the problem-solving process in the digital learning process as the multimodal data.

[0012] The above data analysis unit can convert the above multimodal data into z-scores and generate a multiple regression model using bootstrapping.

[0013] The above data analysis unit can determine at least one digital indicator among stroke acceleration, stroke length, vertical position, tap pressure, and stroke speed.

[0014] The above depression identification unit can build a depression prediction model by learning the correlation between the at least one digital indicator, the number of incorrect answers during the problem-solving process, and the user's PHQ-9 (Patient Health Questionnaire-9) score, and determine whether the user has depression by inputting multimodal data regarding the user's behavior into the depression prediction model.

[0015] Among the embodiments, a depression analysis method based on digital phenotyping is a depression analysis method performed in a depression analysis device, comprising: a step of collecting multimodal data regarding user behavior by monitoring a digital learning process on a user terminal through a data collection unit; a step of detecting at least one digital indicator associated with the user's depressive symptoms by analyzing the multimodal data through a data analysis unit; and a step of determining the degree of the user's depressive symptoms and whether or not there is depression in the digital learning process based on the at least one digital indicator through a depression identification unit.

[0016] The disclosed technology may have the following effects. However, this does not mean that a specific embodiment must include all of the following effects or only the following effects; therefore, the scope of the rights of the disclosed technology should not be understood as being limited by this.

[0017] A digital phenotyping-based depression analysis device and method according to one embodiment of the present invention can screen for depressive symptoms in a user and diagnose and manage depression early by analyzing various digital indicators collected from potential behaviors through monitoring the user's use of a smart device.

[0018] Furthermore, the present invention can diagnose students' mental health problems in the school environment early, improve treatment outcomes through timely intervention, and reduce community healthcare costs, and can provide a foundation for establishing an in-school system that allows for regular screening of students' depression without additional effort by teachers and immediate intervention when necessary.

[0019] FIG. 1 is a diagram illustrating a depression analysis system according to the present invention.

[0020] Figure 2 is a diagram illustrating the functional configuration of the user terminal of Figure 1.

[0021] Figure 3 is a diagram illustrating the system configuration of the depression analysis device of Figure 1.

[0022] Figure 4 is a diagram illustrating the functional configuration of the depression analysis device of Figure 1.

[0023] FIG. 5 is a flowchart illustrating a digital phenotyping-based depression analysis method according to the present invention.

[0024] Figure 6 is a diagram showing an example of a problem solved by students during the data collection process according to the present invention.

[0025] Figure 7 is a diagram illustrating the correlation and multiple regression results between PHQ-9 scores and indicators according to the present invention.

[0026] Figure 8 is a diagram illustrating the correlation and multiple regression analysis results between the PHQ-9 score and the average stroke acceleration according to the present invention.

[0027] The description of the present invention is merely an example for structural or functional explanation, and therefore the scope of the present invention should not be interpreted as being limited by the examples described in the text. That is, since the examples are subject to various modifications and may take various forms, the scope of the present invention should be understood to include equivalents capable of realizing the technical concept. Furthermore, the objectives or effects presented in the present invention do not imply that a specific example must include all of them or only such effects; therefore, the scope of the present invention should not be understood as being limited by them.

[0028] Meanwhile, the meaning of the terms described in this application should be understood as follows.

[0029] Terms such as "first," "second," etc., are intended to distinguish one component from another, and the scope of rights shall not be limited by these terms. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0030] When it is stated that one component is "connected" to another component, it should be understood that it may be directly connected to that other component, or that there may be other components in between. Conversely, when it is stated that one component is "directly connected" to another component, it should be understood that there are no other components in between. Meanwhile, other expressions describing the relationships between components, such as "between" and "exactly between," or "adjacent to" and "directly adjacent to," should be interpreted in the same way.

[0031] A singular expression should be understood to include a plural expression unless the context clearly indicates otherwise, and terms such as "include" or "have" are intended to specify the existence of the implemented features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood not to preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0032] In each step, identifiers (e.g., a, b, c, etc.) are used for convenience of explanation and do not describe the order of the steps; the steps may occur differently from the specified order unless a specific order is clearly indicated in the context. That is, the steps may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

[0033] The present invention may be implemented as computer-readable code on a computer-readable recording medium, and the computer-readable recording medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. Additionally, the computer-readable recording medium may be distributed across networked computer systems, so that computer-readable code can be stored and executed in a distributed manner.

[0034] Unless otherwise defined, all terms used herein have the same meaning as generally understood by those skilled in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having meanings consistent with the context of the relevant technology and should not be interpreted as having an ideal or overly formal meaning unless explicitly defined in this application.

[0035]

[0036] FIG. 1 is a diagram illustrating a depression analysis system according to the present invention.

[0037] Referring to FIG. 1, the depression analysis system (100) may include a user terminal (110), a depression analysis device (130), and a database (150).

[0038] A user terminal (110) may correspond to a terminal device operated by a user. In an embodiment of the present invention, a user may be understood as one or more users, and a plurality of users may be divided into one or more user groups. Each of the one or more users may correspond to one or more user terminals (110). That is, a first user may correspond to a first user terminal, a second user to a second user terminal, ..., and an nth user (where n is a natural number) may correspond to an nth user terminal.

[0039] Additionally, the user terminal (110) may be a computing device that can participate in a digital learning process or a user depression analysis process by linking with the depression analysis device (130) as a device constituting the depression analysis system (100). Here, digital learning may correspond to a learning process in which one participates online in an educational process conducted in the internet space, such as non-face-to-face education or online education. The user terminal (110) may be implemented as a smartphone, laptop, or computer that can be operated in connection with the depression analysis device (130), but is not necessarily limited thereto and may be implemented as various devices including tablet PCs.

[0040] In particular, the user terminal (110) may install and run a dedicated program or application to link with the depression analysis device (130). For example, the user terminal (110) may perform digital learning by participating in a dedicated online learning space provided by the depression analysis device (130), and may provide multimodal data based on the user's behavior during the digital learning process to the depression analysis device (130). Additionally, the digital learning process and data collection operations may be carried out through an interface provided via the dedicated program or application.

[0041] Meanwhile, a user terminal (110) can be connected to a depression analysis device (130) via a network, and multiple user terminals (110) can be connected to the depression analysis device (130) simultaneously.

[0042] The depression analysis device (130) may be implemented as a server corresponding to a computer or program that performs the digital phenotyping-based depression analysis method according to the present invention. Additionally, the depression analysis device (130) may be connected to a user terminal (110) via a wired network or a wireless network such as Bluetooth, WiFi, LTE, etc., and may transmit and receive data with the user terminal (110) through the network.

[0043] Additionally, the depression analysis device (130) may be implemented to operate in connection with an independent external system (not shown in FIG. 1) to perform the digital phenotyping-based depression analysis method according to the present invention. For example, the depression analysis device (130) may operate in conjunction with a learning system that provides online learning, a learning management system that manages learning history, an artificial intelligence system that builds an artificial intelligence model, etc.

[0044] Meanwhile, for the sake of convenience of explanation, the user terminal (110) and the depression analysis device (130) are described as independent devices, but are not necessarily limited thereto, and it is understood that one device may be included in the other device.

[0045] The database (150) may correspond to a storage device that stores various information required during the operation of the depression analysis device (130). For example, the database (150) may store learning materials and learning management information for digital learning or multimodal data collected during the digital learning process, but is not limited thereto, and may store information collected or processed in various forms during the process in which the depression analysis device (130) performs the digital phenotyping-based depression analysis method according to the present invention.

[0046] In addition, in FIG. 1, the database (150) is shown as a device independent of the depression analysis device (130), but it is not necessarily limited thereto and can be implemented as a logical storage device included in the depression analysis device (130).

[0047]

[0048] Figure 2 is a diagram illustrating the functional configuration of the user terminal of Figure 1.

[0049] Referring to FIG. 2, a user terminal (110) can execute a dedicated program or application for digital learning and perform the operation of collecting various user behavior data during the digital learning process. To this end, the user terminal (110) may be implemented to include independent modules for monitoring user behavior and collecting data. Specifically, the user terminal (110) may include a multimodal data collection module (210), a foreground process monitoring module (230), and a web browser monitoring module (250).

[0050] Meanwhile, for the sake of convenience of explanation, the user terminal (110) is described here as being used in a digital learning process, but it is not necessarily limited thereto, and it is obvious that the user terminal (110) can be used in a process of collecting various multimodal data related to the user's behavior in the user's daily life.

[0051] The multimodal data collection module (210) can collect various passive sensor data, keyboard input data, and multimodal data. To this end, the multimodal data collection module (210) can operate in conjunction with various sensors included on the user terminal (110). For example, the multimodal data collection module (210) can periodically collect data information from passive sensors such as angular velocity, acceleration, and light sensors. The multimodal data collection module (210) can collect data such as key value, time, and tap pressure when the user inputs a keyboard, and can collect data when an event occurs, such as the user using a stylus pen. When the user uses a stylus pen, the multimodal data collection module (210) can include a digitizer panel to recognize the position and intensity of the stylus pen and collect data. The data collected by the multimodal data collection module (210) can be stored in internal memory and periodically transmitted to the depression analysis device (130) by the user terminal (110).

[0052] The foreground process monitoring module (230) can perform an operation to detect the execution of a process, and for this purpose, it may include a command to check the package name of the foreground process. The foreground process monitoring module (230) can perform process monitoring as a periodic task, and can collect and record information of the foreground process using the display screen of the user terminal (110) at preset intervals.

[0053] The web browser monitoring module (250) can perform the operation of inspecting information on a web page accessed by a user through a web browser on a user terminal (110). The web browser monitoring module (250) can operate in an event-driven manner in which a processing function is executed whenever an event occurs, and can collect and record text information whenever a user interacts with a text input window (e.g., a search window, an address window, etc.) of the web browser.

[0054]

[0055] Figure 3 is a diagram illustrating the system configuration of the depression analysis device of Figure 1.

[0056] Referring to FIG. 3, the depression analysis device (130) may include a processor (310), memory (330), user input / output unit (350), network input / output unit (370), and communication port unit (390).

[0057] The processor (310) can execute a digital phenotyping-based depression analysis procedure according to an embodiment of the present invention, manage memory (330) that is read or written during this process, and schedule the synchronization time between volatile memory and non-volatile memory in memory (330). The processor (310) can control the overall operation of the depression analysis device (130) and is electrically connected to the memory (330), user input / output unit (350), network input / output unit (370), and communication port unit (390) to control the data flow between them. The processor (310) can be implemented as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit) of the depression analysis device (130).

[0058] The memory (330) may include an auxiliary storage device implemented as non-volatile memory such as an SSD (Solid State Disk) or HDD (Hard Disk Drive) and used to store all data required for the depression analysis device (130), and may include a main memory implemented as volatile memory such as RAM (Random Access Memory). Additionally, the memory (330) may store a set of instructions that execute the digital phenotyping-based depression analysis method according to the present invention by being executed by an electrically connected processor (310).

[0059] The user input / output unit (350) includes an environment for receiving user input and an environment for outputting specific information to the user, and may include an input device including an adapter such as a touch pad, touch screen, virtual keyboard, or pointing device, and an output device including an adapter such as a monitor or touch screen. In one embodiment, the user input / output unit (350) may correspond to a computing device connected via remote access, and in such case, the depression analysis device (130) may be performed as an independent server.

[0060] The network input / output unit (370) provides a communication environment for connecting to a user terminal (110) through a network and may include an adapter for communication such as a LAN (Local Area Network), MAN (Metropolitan Area Network), WAN (Wide Area Network), and VAN (Value Added Network). Additionally, the network input / output unit (370) may be implemented to provide short-range communication functions such as WiFi and Bluetooth, or wireless communication functions of 4G or higher, for wireless transmission of data.

[0061] The communication port section (390) is a hardware interface for connecting to external hardware, for example, the external hardware may include a printer, a mouse, and USB hardware. The communication port section (390) can detect the connection of specific USB hardware to enable it to perform the role of a depression analysis device (130).

[0062]

[0063] Figure 4 is a diagram illustrating the functional configuration of the depression analysis device of Figure 1.

[0064] Referring to FIG. 4, the depression analysis device (130) can perform a digital phenotyping-based depression analysis method according to the present invention. To this end, the depression analysis device (130) may include a data collection unit (410), a data analysis unit (430), a depression identification unit (450), and a control unit (470).

[0065] At this time, embodiments of the present invention are not required to include all of the above components simultaneously; depending on each embodiment, some of the components may be omitted, or some or all of the components may be selectively included. The operation of each component will be described in detail below.

[0066] The data collection unit (410) can collect multimodal data regarding user behavior by monitoring the digital learning process on the user terminal (110). To this end, the data collection unit (410) can operate in conjunction with the user terminal (110) and can receive the collected multimodal data through a dedicated program or application running on the user terminal (110).

[0067] In one embodiment, the data collection unit (410) can collect multimodal data through an application executed on a user terminal (110) during a digital learning process. Here, the multimodal data may include action indicators based on user behavior on the display screen of the user terminal (110). For example, the multimodal data may include data collected from touch or stroke actions using a user's finger or stylus pen on a touchscreen. In particular, the data collection unit (410) can collect various behavioral information generated during the process of a user using a dedicated application for a specific purpose as multimodal data.

[0068] In one embodiment, the data collection unit (410) may collect the user's keyboard-related actions, stylus pen-related actions, and stroke actions collected by the on-screen keyboard sensor and touchscreen sensor of the user terminal (110) during the problem-solving process in the digital learning process as multimodal data. Here, the problem-solving process may correspond to the process in which a problem is provided through the display screen of the user terminal (110), and the user reads the problem and inputs the correct answer. At this time, the user may directly write on the display screen using a stylus pen or input the correct answer using the keyboard displayed on the screen, and the data collection unit (410) may collect multimodal data collected through the display screen among user actions occurring during the problem-solving process or the correct answer input process as an action indicator.

[0069] The data analysis unit (430) can analyze multimodal data to detect at least one digital indicator associated with the user's depressive symptoms. The data analysis unit (430) can extract digital indicators highly associated with the user's depressive symptoms from multimodal data collected during the digital learning process, and can perform preprocessing operations such as missing data processing and denoising during the process. Depressive symptoms refer to various physical, psychological, and emotional symptoms that characterize depression. For example, major symptoms frequently observed when diagnosing depression may include depression or irritability, decreased interest or pleasure, significant increase or decrease in weight or appetite, insomnia or excessive sleep, psychomotor retardation or irritability, loss of energy or fatigue, excessive guilt, loss of concentration, suicidal thoughts, etc. The data analysis unit (430) can detect potential digital indicators indicating at least some of the depressive symptoms (e.g., depression or irritability, psychomotor retardation or irritability, loss of energy or fatigue, loss of concentration).

[0070] Additionally, the data analysis unit (430) can convert multimodal data into z-scores and generate multiple regression models using bootstrapping. First, the data analysis unit (430) can convert indicators including self-reported clinical scores, frequency, duration (ms), length (in pixels), pressure, and ratios using z-scores so that the mean is 0 and the standard deviation is 1. The data analysis unit (430) can compare indicators derived from different distributions through z-score conversion.

[0071] Next, the data analysis unit (430) can perform multiple regression using a bootstrap technique. Bootstrapping may correspond to a method of estimating the sampling distribution by taking multiple samples with replacement from a single random sample. The data analysis unit (430) can construct a multiple regression model through multiple regression and generate a final regression result by removing multicollinearity indicators.

[0072] In one embodiment, the data analysis unit (430) can determine at least one digital indicator among stroke acceleration, stroke length, vertical position, tap pressure, and stroke speed.

[0073] Specifically, stroke acceleration may correspond to the rate of change of velocity during a stroke on a touchscreen, average stroke length may correspond to the average of stroke lengths, and vertical position may correspond to the position between the top and bottom of the screen or the average y-coordinate value during a stroke. Additionally, tap pressure may correspond to the magnitude of the force applied by a finger to tap, and stroke speed may correspond to the number of pixels per second or distance.

[0074] For example, regarding the correlation between depression and digital indicators, the PHQ-9 (Patient Health Questionnaire-9) score, a depression scale, may show a positive correlation with indicators of stroke acceleration, average stroke length, and vertical position, while showing a negative correlation with indicators of tap pressure and stroke speed. Here, the PHQ-9 is a psychometric tool, a self-report questionnaire used to assess depressive symptoms. This questionnaire consists of nine items, each of which is scored from 0 (never) to 3 (almost every day) based on symptoms over the past two weeks. The total score ranges from 0 to 27, with higher scores indicating more severe depressive symptoms. Generally, the interpretation of PHQ-9 score intervals is as shown in Table 1 below.

[0075]

[0076] The data analysis unit (430) can determine digital indicators that have a high correlation with depressive symptoms during the digital learning process through data analysis.

[0077] The depression identification unit (450) can determine the degree and presence of depression in a user during a digital learning process based on at least one digital indicator. For example, the depression identification unit (450) can predict the degree of depression in a user using digital indicators derived by the data analysis unit (430), and can determine whether the user has depression by considering the impact of depression on the digital learning process. Specifically, a high PHQ-9 score is associated with an increase in the average length of strokes and an increase in stroke acceleration, strokes tend to be used frequently near the top of the screen, and may be associated with low tap pressure and slow typing.

[0078] In one embodiment, the depression identification unit (450) can build a depression prediction model by learning the correlation between at least one digital indicator, the number of incorrect answers during the problem-solving process, and the user's PHQ-9 score, and can determine whether the user has depression by inputting multimodal data regarding user behavior into the depression prediction model. Specifically, the depression identification unit (450) can build a depression prediction model by learning learning data including digital indicators collected during the digital learning process, such as stroke acceleration, average stroke length, vertical position, tap pressure, and stroke speed, as well as the number of incorrect answers by the user through problem-solving and the user's PHQ-9 score through a PHQ-9 survey. That is, the depression prediction model may correspond to an artificial intelligence model defined to receive digital indicators associated with the user's depressive symptoms during the digital learning process as input and generate whether the user has depression as output.

[0079] The control unit (470) controls the overall operation of the depression analysis device (130) and can manage the control flow or data flow between the data collection unit (410), the data analysis unit (430), and the depression identification unit (450).

[0080]

[0081] FIG. 5 is a flowchart illustrating a digital phenotyping-based depression analysis method according to the present invention.

[0082] Referring to FIG. 5, the depression analysis device (130) can collect multimodal data regarding user behavior by monitoring a digital learning process on a user terminal (110) through a data collection unit (410) (step S510). The depression analysis device (130) can detect at least one digital indicator associated with the user's depressive symptoms by analyzing the multimodal data through a data analysis unit (430) (step S530).

[0083] Additionally, the depression analysis device (130) can build a depression prediction model by learning at least one digital indicator through the depression identification unit (450) (step S550). The depression analysis device (130) can determine the degree and presence of depression in the user during the digital learning process based on at least one digital indicator through the depression identification unit (450) (step S570).

[0084]

[0085] Figure 6 is a diagram showing an example of a problem solved by students during the data collection process according to the present invention.

[0086] Figure 6 was based on 927 first-year middle school students with an average age of 13. Of these, 483 (52.2%) were female students. Students with intellectual disabilities or autism spectrum disorders, central nervous system disorders such as epilepsy or severe head injury, serious health problems that could cause psychological symptoms, or those unable to use smart devices were excluded. Information on the demographic and clinical characteristics of the students included in the data collection is shown in Table 2 below.

[0087]

[0088] The students for data collection first completed the PHQ-9, then solved 20 Korean language subjective questions for 45 minutes in accordance with the first-year middle school curriculum, took a 15-minute break, and then solved 20 mathematics subjective questions for 45 minutes. During this time, they were allowed to freely use keyboards and digital styluses as input tools.

[0089] Figure 6 (a) is an example of a Korean language problem with an answer entered using a keyboard, and (b) is an example of a math problem with an answer entered using a stylus pen.

[0090] Digital data can be collected using two passive sensors built into the tablet computer, the terminal device: a screen keyboard sensor and a touchscreen sensor, while students are solving problems. At this time, through the Dr. Simon module pre-installed on the tablet, 77 unique types of interactions of the keyboard and stylus, such as keystroke count, speed, acceleration, length, duration, trajectory, and pressure, can be captured and recorded inconspicuously at 24 Hz (or every 42.67 ms) during learning activities.

[0091] Using the PHQ-9 questionnaire, the measurement of various indicators, including self-reported clinical scores, frequency, duration, length, pressure intensity, and ratio, in which students self-assess depressive symptoms, can be standardized through z-score normalization.

[0092]

[0093] Figure 7 is a diagram illustrating the correlation and multiple regression results between PHQ-9 scores and indicators according to the present invention.

[0094] In the case of Figure 7, it can be confirmed that average keystroke acceleration, number of incorrect answers, average keystroke length, average tap pressure, and average keystroke speed are predictors of the depression symptom score. Specifically, it can be seen that keystroke acceleration has the strongest positive correlation with depression symptoms with a correlation coefficient of 0.248, the number of incorrect answers also has a positive correlation with depression symptoms with a correlation coefficient of 0.143, and keystroke length also has a positive correlation with depression symptoms with a correlation coefficient of 0.116. This means that the higher the keystroke acceleration, the more incorrect answers, and the longer the keystroke length, the higher the PHQ-9 score. It can be seen that tap pressure has a negative correlation with depression symptoms with a correlation coefficient of -0.076, and keystroke speed also has a negative correlation with depression symptoms with a correlation coefficient of -0.171. This means that the lower the tap pressure and the lower the stroke speed, the higher the PHQ-9 score. Here, R 2 The value was found to be 0.0252.

[0095]

[0096] Figure 8 is a diagram illustrating the correlation and multiple regression analysis results between the PHQ-9 score and the average stroke acceleration according to the present invention.

[0097] In Figure 8, the vertical axis represents the PHQ-9 score, indicating the degree of depressive symptoms; higher values ​​signify more severe symptoms. The horizontal axis represents stroke acceleration, a value indicating the acceleration of a stroke when using a stylus or typing. Here, each point represents a pair of students' PHQ-9 scores and stroke acceleration values. The red trend line visually illustrates the relationship between stroke acceleration and PHQ-9 scores. Digital data variables describing the PHQ-9 score may include the number of incorrect answers, stroke length, speed and acceleration, average vertical stroke position, and average pressure applied while typing on the keyboard. Higher PHQ-9 scores were associated with lower tap pressure, a higher number of incorrect answers, and more frequent use of strokes upward on the tablet. Additionally, the PHQ-9 score decreased as the average stroke length increased, the average stroke speed decelerated, and the average keystroke acceleration increased.

[0098] The results of the regression analysis indicated that higher tap pressure was associated with lower PHQ-9 scores. Lack of energy and lethargy are typical symptoms of depression. These depressive symptoms can affect tap pressure because muscle tension increases in stressful situations. Although stress and depressive states are distinct, the correlation between tap pressure and mental symptoms mediated by muscle tension may be significant. Depressive symptoms can manifest as psychomotor agitation or psychomotor retardation. In adolescents, psychomotor retardation is observed more frequently in the presence of depressive symptoms, which may contribute to the negative correlation between tap pressure and depressive symptoms.

[0099] Regression analysis revealed that higher PHQ-9 scores were associated with a higher number of incorrect answers and a greater distribution of strokes toward the top of the tablet. Depression is known to be closely associated with and cause cognitive impairments, such as difficulty concentrating. In particular, the decline in students' academic achievement is a well-documented issue associated with depression. Therefore, the positive correlation between the number of incorrect answers and depression scores is consistent with established knowledge.

[0100] Furthermore, children and adolescents often exhibit depressive symptoms without feeling sadness, and these symptoms may frequently be overlooked or reported inappropriately. This invention can establish a digital phenotype of depression in adolescents by analyzing the correlation between digital data passively collected in a daily academic environment and the manifestation of depressive symptoms. This enables educators to support the timely identification and resolution of students' depressive symptoms.

[0101]

[0102] Although the present invention has been described above with reference to preferred embodiments, those skilled in the art will understand that various modifications and changes can be made to the invention without departing from the spirit and scope of the invention as described in the following claims.

Claims

1. A data collection unit that monitors a digital learning process on a user terminal and collects multimodal data regarding behaviors related to the use of the user's finger or stylus pen; A data analysis unit that analyzes the above multimodal data to detect at least one digital indicator associated with the user's depressive symptoms; and A digital phenotyping-based depression analysis device comprising: a depression identification unit that determines the degree of depressive symptoms and whether the user has depression during the digital learning process based on at least one digital indicator.

2. In paragraph 1, the data collection unit During the digital learning process described above, the multimodal data is collected through an application executed on the user terminal, and A digital phenotyping-based depression analysis device characterized in that the above multimodal data includes operation indicators based on user behavior on the display screen of the user terminal.

3. In paragraph 2, the data collection unit A digital phenotyping-based depression analysis device characterized by collecting the user's keyboard or stylus pen-related movements and stroke movements, collected by the on-screen keyboard sensor and touchscreen sensor of the user terminal during the problem-solving process in the digital learning process, as multimodal data.

4. In paragraph 1, the data analysis unit A digital phenotyping-based depression analysis device characterized by converting the above multimodal data into z-scores and generating a multiple regression model using bootstrapping.

5. In paragraph 4, the above data analysis unit A digital phenotyping-based depression analysis device characterized by determining at least one digital indicator among stroke acceleration, stroke length, vertical position, tap pressure, and stroke speed.

6. In paragraph 3, the depression identification unit A digital phenotyping-based depression analysis device characterized by building a depression prediction model by learning the correlation between at least one digital indicator, the number of incorrect answers during the problem-solving process, and the user's PHQ-9 (Patient Health Questionnaire-9) score, and inputting multimodal data regarding the user's behavior into the depression prediction model to determine whether the user has depression.

7. In a method for analyzing depression performed in a depression analysis device, A step of collecting multimodal data regarding user behavior by monitoring the digital learning process on the user terminal through a data collection unit; A step of detecting at least one digital indicator associated with the user's depressive symptoms by analyzing the multimodal data through a data analysis unit; and A digital phenotyping-based depression analysis method comprising: a step of determining the degree of depressive symptoms and whether the user has depression in the digital learning process based on at least one digital indicator through a depression identification unit.

8. In Paragraph 7, The step of collecting the above data is, During the digital learning process described above, the multimodal data is collected through an application executed on the user terminal, and A digital phenotyping-based depression analysis method characterized by the above multimodal data including operation indicators based on user behavior on the display screen of the user terminal.

9. In Paragraph 8, The step of collecting the above data is, A digital phenotyping-based depression analysis method characterized by collecting the user's keyboard or stylus pen-related movements and stroke movements, collected by the on-screen keyboard sensor and touchscreen sensor of the user terminal during the problem-solving process in the digital learning process, as multimodal data.

10. In Paragraph 7, The step of detecting the above digital indicator is, A digital phenotyping-based depression analysis method characterized by converting the above multimodal data into z-scores and generating a multiple regression model using bootstrapping.

11. In Paragraph 10, The step of detecting the above digital indicator is, A digital phenotyping-based depression analysis method characterized by determining at least one digital indicator among stroke acceleration, stroke length, vertical position, tap pressure, and stroke speed.

12. In Paragraph 9, The step of determining whether there is depression mentioned above is, A digital phenotyping-based depression analysis method characterized by building a depression prediction model by learning the correlation between at least one digital indicator, the number of incorrect answers during the problem-solving process, and the user's PHQ-9 (Patient Health Questionnaire-9) score, and inputting multimodal data regarding the user's behavior into the depression prediction model to determine whether the user has depression.

13. In a computer-readable recording medium on which a program for executing a method for analyzing depression is recorded, The above method for analyzing depression is, A step of collecting multimodal data regarding user behavior by monitoring the digital learning process on the user terminal; A step of analyzing the multimodal data to detect at least one digital indicator associated with the user's depressive symptoms; and A computer-readable recording medium comprising: a step of determining the degree of depressive symptoms and whether the user has depression in the digital learning process based on at least one digital indicator.

Images