Method for assisting in early warning of depression through digital writing

Abstract

The invention relates to the technical field of emotion recognition, in particular to a digitized writing assisted depression early warning method, which comprises the following steps: a user completes a specified task on dot matrix paper by using a dot matrix digital pen; data acquisition: acquiring real-time information data when a user writes through a dot matrix digital pen; processing data; establishing a training model; and outputting a result by the auxiliary early warning model: inputting the normalized handwriting features into a pre-trained deep learning model based on a fusion attention mechanism, and outputting an early warning result of whether the user has depression or not by the model. According to the invention, the problems of harsh acquisition conditions and complex flow in the existing auxiliary early warning scheme are solved, and most physical hardware equipment and physiological signal detection equipment are omitted; characteristic recognition of the handwriting is more accurate, and an early warning result is more reliable.

Description

Technical Field

[0001] The present invention relates to the technical field of emotion recognition, and particularly to a method for digitally writing-assisted early warning of depression. Background Art

[0002] With the rapid development of artificial intelligence technology, more and more intelligent products are used in people's life and study, especially in the field of affective computing which is in rapid development. With the continuous development of modern society, mental health and psychological counseling services have also received increasing attention. In particular, the student group, which belongs to the high-intensity and high-pressure population, needs special attention to their emotional state. At present, in various fields, methods such as asking and empirical judgment are generally used to obtain the emotional state of humans, but the accuracy of the information obtained is often limited by the honesty of the interviewee and the professionalism of the judge. In order to be able to timely pay attention to the emotions during the writing process, track abnormal changes in students' emotions, prevent depression or harm to mental health, timely adjust the mood, and overcome emotional distress brought by anxiety, tension, etc., it is very meaningful to improve the mental health early warning of students.

[0003] Emotion recognition technology refers to that AI automatically discriminates the emotional state of an individual by obtaining the physiological or non-physiological signals of the individual, which is an important part of affective computing. The research content of emotion recognition includes aspects such as facial expression, speech, heart rate, behavior, text, and physiological signal recognition, and judges the emotional state of the user through the above content.

[0004] In the prior art, a patent with the application number of 202111171615.7 discloses a handwriting analysis method. This method relies on deep learning and machine vision technologies to train a handwriting feature recognition model based on a large number of handwriting samples and annotation files. At the same time, a knowledge graph of handwriting features and personality traits is established based on a large amount of data analysis. In the application stage, the handwriting features are obtained by calling the pre-trained handwriting feature recognition model, and then the corresponding basic personality traits and emotional state of the testee are obtained from the knowledge graph of handwriting features and personality traits. Then, the vertical index evaluation model is called to fuse the basic personality traits and emotional state to determine the performance tendency or ability specialty of the writer during writing. Then, the text generation model is called to organize sentences according to the evaluation results, and an evaluation analysis report for different dimensions and application scenarios of the writer can be generated. This prior art requires obtaining the handwriting picture of the person to be detected as the target picture, which can only provide static information of the handwriting and cannot obtain the real-time dynamic handwriting features of the detector; in addition, this prior art requires a large number of handwriting samples and annotation files, and the sample collection is difficult.

[0005] In the prior art, a method for digitally writing and recognizing emotional states is disclosed in a patent with the application number 202110961674.8. The method includes the following steps: The user writes text on a dot matrix paper with a dot matrix digital pen; real-time information data during the user's writing is obtained; the obtained real-time information data is processed to obtain handwriting features that are more closely related to emotional labels, and the obtained handwriting features that are more closely related to emotional labels are subjected to data normalization; the normalized data of the handwriting features is input into a pre-trained emotion recognition model to obtain the user's emotional state category. The present invention solves the problem of complex and demanding information collection conditions in existing emotion recognition solutions. After using the font feature method to recognize the font features written by the user, emotion state classification is performed, so most physical hardware devices and physiological signal detection devices are eliminated. This prior art solution can only perform general emotion recognition and does not specifically analyze the handwriting features of depression patients. Therefore, the specific handwriting features of depression patients have not been extracted; in addition, this prior art cannot warn depression patients, lacks pertinence, and fails to achieve the purpose of warning depression. Summary of the Invention

[0006] The present invention provides a method for digitally writing to assist in warning of depression for the problems of the prior art, solving the problems of harsh collection conditions and complex processes in existing auxiliary warning solutions. Since the subject can be judged whether they may have depression as long as they complete six writing tasks, most physical hardware devices and physiological signal detection devices are eliminated; at the same time, digital writing technology can obtain dynamic information such as the user's real-time pen-down coordinates, pressure, and time, and the feature recognition of handwriting is more accurate, and the warning result is more reliable.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0008] The present invention provides a method for digitally writing to assist in warning of depression, which includes the following steps:

[0009] Step S1: The user uses a dot matrix digital pen to complete a specified task on a dot matrix paper;

[0010] Step S2: Data collection: Obtain real-time information data during the user's writing through the dot matrix digital pen;

[0011] Step S3: Data processing: Process the obtained real-time information data; extract handwriting features related to the label of depression patients, and perform data normalization processing on the extracted handwriting features;

[0012] Step S4, establishing a training model: Divide the labeled sample set into a training sample set and a test sample set. Use the training sample set to train a deep learning model based on a fusion attention mechanism, and use the test sample set to test the model. After optimizing and adjusting various parameters, if the classification accuracy reaches the preset requirement, the auxiliary warning model is obtained.

[0013] Step S5, output result of the auxiliary warning model: Input the normalized handwriting features into the pre-trained deep learning model based on the fusion attention mechanism, and the model outputs the warning result of whether the user has depression.

[0014] Preferably, there are six specified tasks completed by the user using the dot matrix digital pen on the dot matrix paper in step S1. The six specified tasks include two drawing tasks and four writing tasks.

[0015] Preferably, the two drawing tasks include drawing intersecting pentagons once or multiple times and drawing one or more clock diagrams with time.

[0016] Preferably, the four writing tasks include writing a line of lowercase English letters, writing a line of uppercase English letters, writing a line of neutral Chinese text, and writing a single Chinese character multiple times.

[0017] Preferably, the real-time information data includes the two-dimensional coordinates, pressure values, timestamps, pen states, and stroke counts of each point passed by the pen tip when writing with the dot matrix digital pen.

[0018] Preferably, the handwriting features include writing speed, acceleration, pressure, and deviation.

[0019] Preferably, in step S3, extract the handwriting features related to the depression patient label, and use the stepwise regression method to screen the dynamic handwriting features closely related to the depression label. When establishing the regression model, gradually add or delete features to find the best feature combination, so as to obtain the optimal prediction model.

[0020] Preferably, in step S4, use the method based on gradient descent to optimize and adjust various parameters.

[0021] Preferably, in step S4, the classification accuracy is preset to be above 80%.

[0022] Advantages of the present invention:

[0023] The present invention provides a method for digitally writing-assisted early warning of depression for the problems of the prior art, and solves the problems of harsh acquisition conditions and complex processes in the existing assisted early warning solutions. Since the subject can be judged whether they may have depression as long as they complete six writing tasks, most physical hardware devices and physiological signal detection devices are eliminated. At the same time, the digital writing technology can obtain dynamic information such as the real-time pen-down coordinates, pressure, and time of the user, and the feature recognition of the handwriting is more accurate, and the early warning result is more reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flowchart of a method for digitally writing-assisted early warning of depression according to the present invention.

[0025] Figure 2 It is a schematic diagram of two drawing tasks and four writing tasks of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] For the convenience of those skilled in the art to understand, the present invention will be further described below in conjunction with the embodiments and the drawings. The content mentioned in the embodiments is not a limitation to the present invention. The present invention will be described in detail below with reference to the drawings.

[0027] Embodiment 1

[0028] Embodiment 1 of the present application provides a method for digitally writing-assisted early warning of depression, as Figure 1 shown, which includes the following steps:

[0029] Step S1, the user uses a dot matrix digital pen to complete the specified tasks on the dot matrix paper; among them, the user takes clinically depressed patients, individuals with high subclinical depression characteristics, and healthy individuals as the tested subjects respectively; there are six specified tasks that the user uses a dot matrix digital pen to complete on the dot matrix paper, as Figure 2 shown, the six specified tasks include two drawing tasks and four writing tasks; among them, the two drawing tasks include drawing intersecting pentagons once or multiple times and drawing one or more clock diagrams with time. Preferably, the four writing tasks include writing a line of lowercase English letters, writing a line of uppercase English letters, writing a line of neutral Chinese text, and writing a single Chinese character (such as Figure 2(in the personal information section) multiple times. Step S2, Data acquisition: Obtain real-time information data during the user's writing through a dot matrix digital pen; among them, the "dot matrix" is composed of some very small dots arranged according to special algorithm rules; the role of the dot matrix is to provide a coordinate parameter information to the dot matrix digital pen to ensure that when the dot matrix digital pen writes on the digital device, it can accurately record the writing strokes; according to the characteristics of the dot matrix digital pen, when the tip of the pen is pressed down, the pressure sensor is triggered to start the built-in high-speed camera, which takes pictures of the dot matrix passed by the tip of the pen at a speed of hundreds of times per second; the real-time information data includes the two-dimensional coordinates, pressure values, timestamps, pen status, and number of strokes of each point passed by the tip of the pen when writing with the dot matrix digital pen;

[0030] Step S3, Data processing: Process the obtained real-time information data; extract the handwriting features related to the labels of depression patients, and perform data normalization processing on the extracted handwriting features;

[0031] Step S4, Establish a training model: Divide the labeled sample set into a training sample set and a test sample set, use the training sample set to train a deep learning model based on the fusion attention mechanism, and use the test sample set for model testing; after optimizing and adjusting various parameters, if the classification accuracy reaches the preset requirement, the auxiliary early warning model is obtained; among them, the method based on gradient descent is used to optimize and adjust various parameters; preferably, the classification accuracy is preset to be above 80%;

[0032] Step S5, Output the result of the auxiliary early warning model: Input the normalized handwriting features into the pre-trained deep learning model based on the fusion attention mechanism, and the model outputs the early warning result of whether the user has depression.

[0033] The design of the present invention is ingenious and has the following advantages: The early warning process is simple and convenient, and the early warning can be carried out only by collecting and analyzing handwriting data; it has high accuracy and reliability, and can provide an important auxiliary means for the early warning of depression.

[0034] In the embodiment of the present application, the handwriting features include but are not limited to features such as writing speed, acceleration, pressure, and deviation, and the specific features are shown in Table 1:

[0035] Table 1 List of 15 categories of handwriting features of depression patients

[0036]

[0037] Note: No. is the feature category number

[0038] In the embodiments of the present application, the continuous stroke degree (Lpen) is the ratio of the number of missing strokes to the total number of strokes in the task, which can be expressed by Equation (1): The number of strokes in the entire writing task is the number of strokes actually written by the subject (Wnum). The number of missing strokes is the total number of strokes (Nstroke) in the task minus the number of strokes actually written.

[0039]

[0040] The average pressure is expressed by Equation (2): The writing pressure is the pressure-sensitive data obtained from the 1024-level pressure sensor built into the dot matrix pen; a higher pressure-sensitive value means a higher writing pressure.

[0041]

[0042] The entropy value is expressed by Equation (3): Information entropy is often used to characterize the uncertainty of information. The lower the entropy value, the higher the self-similarity of the signal and the lower the complexity.

[0043]

[0044] The airtime is expressed by Equation (4): It refers to the time between the end of one stroke and the start of the next stroke, recorded as the airtime (ti) between strokes. The airtime of this task is the sum of the airtimes between all strokes.

[0045]

[0046] The variance is expressed by Equation (5): It reflects the degree of dispersion of a set of data, that is, the degree of deviation. M is the average value.

[0047]

[0048] The average speed is expressed by Equation (8): It reflects the writing rate, the average speed of the entire task. This is calculated by the stroke length expressed by Equation (6) and the time taken for writing expressed by Equation (7).

[0049]

[0050]

[0051]

[0052] In the embodiments of the present application, the stepwise regression method is used to screen the dynamic handwriting features that are more closely related to the depression label, such as: inclination, pressure, acceleration, acceleration in the x-axis direction, acceleration in the y-axis direction, minimum acceleration, variance of x coordinate points, variance of y coordinate points and other features; the stepwise regression method is a regression analysis method for gradually selecting features, which is used to determine which features are the most important for predicting the target variable. When establishing a regression model, it gradually adds or deletes features to find the best combination of features, so as to obtain the optimal prediction model. Specifically, the backward elimination method is adopted: starting from the model containing all features, gradually eliminate features until no further improvement can be made; the steps are as follows: A1. Initialize the model containing all features; A2. For each feature in the current model, remove it from the model respectively to obtain a new model; A3. Select the feature that makes the performance of the new model optimal, and remove this feature from the current model; A4. Repeat step A3 until the performance of the model can no longer be improved. Data normalization (z-score) is performed on these obtained features to converge the data to the range of [0, 1], so as to eliminate the adverse effects caused by singular sample data. The data normalization formula: X* = (x - μ) / σ, where: X* is the normalized value of the data point, x is the original value of the data point, μ is the mean of the data, and σ is the standard deviation of the data.

[0053] In the embodiments of the present application, in step S3, handwriting features related to the depression patient label are extracted, and the stepwise regression method is used to screen the dynamic handwriting features that are closely related to the depression label; features are gradually added or deleted when establishing a regression model to find the best combination of features, so as to obtain the optimal prediction model.

[0054] In the embodiments of the present application, the labeled sample set is divided into a training sample set and a test sample set. The training sample set is used to train the deep learning (SimAm-TabNet) algorithm based on the fusion attention mechanism, and the test sample set is used to test the model. After optimizing and adjusting various parameters using the gradient descent method, if the classification accuracy meets the requirements (the classification accuracy is above 80%), an available auxiliary early warning model is obtained. The deep learning algorithm SimAm-TabNet based on the fusion attention mechanism combines the attention mechanism and the TabNet framework to process the diverse and complex number of handwriting in the present invention. SimAm-TabNet has a fast learning process and high accuracy. It can evaluate the importance of variables and finally obtain the depression label. Similar to the random forest, SimAm-TabNet is a classifier based on multiple decision trees, and by introducing the attention mechanism, the model can focus more on key features. This deep learning algorithm can effectively handle the complex sentiment classification tasks faced in the present invention. The deep learning model in the embodiments of the present application, TabNet, is a neural network specifically for tabular data. It uses the idea of sequential attention to simulate the behavior of decision trees. It can be regarded as a multi-step neural network, using two key operations in each step: the attention transformer and the feature transformer. This model can achieve end-to-end learning, while selecting and processing the most valuable features, thereby improving interpretability and learning ability. The feature data enters the BN (batch normalization) layer for data standardization.

[0055] The results are used for feature calculation and enter the feature selection part for weight assignment; important features are filtered out through the weight matrix, and then segmented analysis is performed again. The part after passing through the activation function is used as a common standard. After completing all steps, the common features are used to complete the final decision. Another part enters the next step to learn the individual features of each step. And so on until all decision steps are completed. SimAm is selected as the attention mechanism in TabNet, which is a non-referential attention mechanism module. To better implement attention, it is necessary to evaluate the importance of each neuron and define an energy function to minimize this energy function. The smaller the energy, the greater the difference between the target neuron and other neurons, and the higher the importance. The minimum energy formula is shown in (9):

[0056]

[0057] The importance of the neuron is (1 / e*). After obtaining the importance, the feature matrix is enhanced using equation (10):

[0058]

[0059] The SimAm-TabNet adopted in the present invention is an improved version based on the original TabNet, with the attention mechanism SimAm added between feature selection and feature calculation to enable TabNet to obtain better target weights from feature weights. The parameter settings of SimAm-TabNet are as follows: The number of neurons in the input layer: The number of incoming neurons is equivalent to the number of features, set to 48. The number of neurons in the output layer: Equal to the number of categories to be distinguished, with a value of 2 for binary classification tasks. The number of decision steps: Used to set the size of the network structure steps, set to 8. The number of features in the prediction phase: Equal to the number of features input in each decision step, set to 30. The number of features in the feature selection phase: Equal to the number of features output in each decision step, set to 10. The feature selection phase attention update ratio: Equal to the ratio of setting feature attention update in feature selection; set to 1.3. Batch size: Set to 200. Approximately one-twentieth of the total data volume. Optimizer: Set to the Adam optimizer. Learning rate: The learning rate of SimAm-TabNet is set to 0.001. Epochs: Set to 500.

[0060] In addition, the loss function of the SimAm-TabNet model is: cross entropy.

[0061] The present invention provides a method for digitally writing-assisted early warning of depression for the problems of the prior art, solving the problems of harsh acquisition conditions and complex processes in the existing assisted early warning solutions. Since the subject can be judged whether they may have depression as long as they complete six writing tasks, most physical hardware devices and physiological signal detection devices are eliminated; at the same time, the digital writing technology can obtain dynamic information such as the user's real-time pen-down coordinates, pressure, time, etc., and is more accurate in identifying the characteristics of the handwriting, the early warning result is more reliable, and the early warning process is simple and convenient. The early warning can be carried out only by collecting and analyzing the handwriting data.

[0062] In the embodiments of the present application, the trained SimAm-TabNet model can more accurately assist in the early warning of depression.

[0063] Embodiment 2

[0064] In Embodiment 2 of the present application, in terms of digital writing, a tablet computer and a digital tablet can be used for training data collection. When training the depression-assisted early warning model, not only the random forest algorithm can be used for classification, but also other classification algorithms of machine learning can be used for classification, such as support vector machine, k-nn (K-nearest neighbor), etc. However, the accuracy of the trained model is not as high as that of the SimAm-TabNet model in Embodiment 1.

[0065] The above are only the preferred embodiments of the present invention, and there is no limitation to the present invention in any form. Although the present invention is disclosed above in the preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art, without departing from the scope of the technical solution of the present invention, when making some changes or modifications using the above-disclosed technical content as equivalent embodiments of equivalent changes, but as long as it does not depart from the content of the technical solution of the present invention, any simple modification, equivalent change and modification made to the above embodiments according to the technical solution of the present invention shall fall within the scope of the technical solution of the present invention.

Claims

1. A method for digitally assisted writing to warn of depression, characterized in that It includes the following steps: Step S1: The user uses a dot matrix digital pen to complete a specified task on dot matrix paper; Step S2: Data acquisition: Obtain real-time information data during the user's writing through the dot matrix digital pen; Step S3: Data processing: Process the obtained real-time information data; Extract handwriting features related to the label of depression patients, and perform data normalization on the extracted handwriting features; Step S4: Establish a training model: Divide the labeled sample set into a training sample set and a test sample set, use the training sample set to train a deep learning model based on the fusion attention mechanism, and use the test sample set for model testing; After optimizing and adjusting various parameters, if the classification accuracy reaches the preset requirement, the auxiliary warning model is obtained; Step S5: Output result of the auxiliary warning model: Input the normalized handwriting features into the pre-trained deep learning model based on the fusion attention mechanism, and the model outputs the warning result of whether the user has depression.

2. The method for digitally assisted writing to warn of depression according to claim 1, wherein: There are six specified tasks completed by the user using the dot matrix digital pen on the dot matrix paper in Step S1. The six specified tasks include two drawing tasks and four writing tasks.

3. A method for digitally writing-assisted early warning of depression according to claim 2, characterized in that: The two drawing tasks include drawing intersecting pentagons once or multiple times and drawing one or more clock diagrams with time.

4. A method for digitally assisted writing to warn of depression according to claim 2, characterized in that: The four writing tasks include writing a line of lowercase English letters, writing a line of uppercase English letters, writing a line of neutral Chinese text, and writing a single Chinese character multiple times.

5. A method for digitally assisted writing to warn of depression according to claim 1, characterized in that: The real-time information data includes the two-dimensional coordinates of each point passed by the pen tip, pressure value, timestamp, pen status, and number of strokes when writing with the dot matrix digital pen.

6. A method for digitally assisted writing to warn of depression according to claim 1, characterized in that: The handwriting features include writing speed, acceleration, pressure, and deviation.

7. A method for digitally assisted writing to warn of depression according to claim 1, characterized in that: In Step S3, extract handwriting features related to the label of depression patients, and use the stepwise regression method to screen dynamic handwriting features closely related to the depression label; Add or delete features step by step when establishing the regression model to find the best feature combination, so as to obtain the optimal prediction model.

8. A method for digitally assisted writing to warn of depression according to claim 1, characterized in that: In Step S4, use the method based on gradient descent to optimize and adjust various parameters.

9. A method for digitally writing-assisted early warning of depression according to claim 1, characterized in that: In Step S4, the classification accuracy is preset to be above 80%.

Images