Model training method, method for mental health detection, terminal and readable storage medium

Abstract

The present application discloses a model training method, a method for mental health detection, a terminal and a readable storage medium, wherein the model training method includes: obtaining a user psychological data set to be trained, and extracting a feature representation sequence of each sample in the user psychological data set, wherein the user psychological data set includes data sets of multiple categories; training an improved BERT model according to the feature representation sequence to obtain a pre-trained language model for mental health detection; wherein the BERT model loss function is improved to a cross-entropy weighted focal loss function. Using the BERT model as the basic model and improving the BERT model loss function to a cross-entropy weighted focal loss function can not only improve the generalization ability of the pre-trained language model on various tasks, but also optimize the deviation caused by the imbalance of input data, so as to improve the performance of the pre-trained language model as a whole, so that when the pre-trained language model is used for mental health detection, accurate detection results can be obtained.

Description

Technical Field

[0001] The present application relates to the technical field of deep learning models, and in particular to a model training method, a method for mental health detection, a terminal and a readable storage medium. Background Art

[0002] With the rapid development of digitalization and informatization, people's pace of life has accelerated, and the mental health problems brought about by it have become increasingly prominent. In the busy and complicated daily life, people may ignore the importance of mental health, resulting in some potential psychological problems not being discovered and dealt with in time. Therefore, it is particularly important to monitor personal mental health in a timely manner.

[0003] At present, mental health monitoring methods rely on the diagnosis and evaluation of professionals such as psychologists and psychologists, which have certain effects. However, since the development of mental health problems is gradual, many psychological problems such as depression and anxiety may not show obvious external symptoms in the early stages. When the symptoms are obvious, the individual is often already in a more serious stage of the problem. However, professional diagnosis is usually only carried out when the symptoms are more significant, which will cause the best window for intervention and treatment to be missed. In addition, traditional diagnosis relies on regular interviews, questionnaires and other tools. These assessments are often staged, and the individual's mental state will change over time and situation. Therefore, the time point of the assessment may not be completely connected with the actual situation, which affects the timely monitoring and intervention of mental health problems. This shows that the above-mentioned methods of diagnosis and evaluation by professionals have lags and limitations, resulting in inaccurate diagnostic results.

[0004] Therefore, there is an urgent need to provide a model training method, a method for mental health testing, a terminal and a readable storage medium to improve the accuracy of mental health testing.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The embodiments of the present application aim to solve the technical problem that the diagnosis and evaluation methods performed by professionals have lags and limitations, resulting in insufficient accuracy of the diagnosis results, by providing a model training method, a method for mental health testing, a terminal, and a readable storage medium.

[0007] To achieve the above-mentioned purpose, an embodiment of the present application provides a model training method, the model training method comprising: obtaining a user psychological data set to be trained, and extracting a feature representation sequence of each sample in the user psychological data set, wherein the user psychological data set includes data sets of multiple categories; training an improved BERT model according to the feature representation sequence to obtain a pre-trained language model for mental health detection; wherein the loss function of the BERT model is improved to a cross-entropy weighted focal loss function, and the cross-entropy weighted focal loss function is:

[0008]

[0009] Wherein, G(y, p) represents the loss function value, i represents the index of the category, N represents the number of samples, and t represents the weight parameter. γ represents the hyperparameter, y i represents the true probability that the sample belongs to the i-th category, p i represents the model prediction probability that the sample belongs to the i-th category, p k represents the output probability.

[0010] Optionally, the step of obtaining the user psychological dataset to be trained and extracting the feature representation sequence of each sample in the user psychological dataset includes: obtaining the user psychological dataset to be trained; and extracting the feature representation sequence of each sample in the user psychological dataset using the encoder of the BERT model.

[0011] Optionally, the step of using the encoder of the BERT model to extract the feature representation sequence of each sample in the user psychological dataset includes: generating a corresponding tag embedding representation based on the tag information of the word itself in the sample; generating a corresponding position embedding representation based on the position of the word in the sentence; generating a corresponding fragment embedding representation based on the sentence or paragraph position of the word; integrating the tag embedding representation, the position embedding representation and the fragment embedding representation to generate a feature representation of the word; and combining the feature representations of all the words in the sample into the feature representation sequence.

[0012] Optionally, after the step of training the improved BERT model according to the feature representation sequence to obtain a pre-trained language model for mental health detection, the model training method further comprises: obtaining a dataset for a specific task, wherein the dataset is annotated with input labels and output labels;

[0013] The pre-trained language model is trained according to the data set of the specific task to obtain a model for mental health detection; wherein the model parameter optimization algorithm of the pre-trained language model is improved to the LORA algorithm.

[0014] In addition, to achieve the above-mentioned purpose, an embodiment of the present application provides a method for mental health testing, which includes: obtaining user psychological data of the user to be tested; inputting the user psychological data of the user to be tested into a model for mental health testing obtained by training the model using the training method of the model, and obtaining the mental health testing result of the user to be tested.

[0015] Optionally, the user psychological data is psychological data of the user to be detected recorded continuously for a preset period of time.

[0016] In addition, to achieve the above-mentioned purpose, the present application also provides a terminal device, which includes: a memory, a processor, and a training program for a model of mental health detection of the terminal device stored in the memory and runable on the processor, and / or a program for mental health detection of the terminal device stored in the memory and runable on the processor; wherein, when the training program for the model of mental health detection of the terminal device is executed by the processor, the steps of the model training method as in the first aspect and any one of its embodiments are implemented, and when the program for mental health detection of the terminal device is executed by the processor, the steps of the method for mental health detection as in any one of the second aspect and any one of its embodiments are implemented.

[0017] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which is stored a training program for a model for mental health detection of a terminal device and / or a program for mental health detection; wherein, when the training program for a model for mental health detection of the terminal device is executed by a processor, the steps of the model training method as in the first aspect and any one of its embodiments are implemented, and when the program for mental health detection of the terminal device is executed by a processor, the steps of the method for mental health detection as in the second aspect and any one of its embodiments are implemented.

[0018] This application has at least the following beneficial effects:

[0019] This application uses the BERT model as the basic model and improves the BERT model loss function to a cross-entropy weighted focal loss function, which can not only improve the generalization ability of the pre-trained language model on various tasks, but also optimize the deviation caused by the imbalance of input data, so as to improve the overall performance of the pre-trained language model, so that when the pre-trained language model is used for mental health testing, accurate test results can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present application.

[0021] Figure 2A flowchart of the steps of Example 1 of a model training method provided in this application;

[0022] Figure 3 A flowchart of the steps of Example 2 of a model training method provided in this application;

[0023] Figure 4 A flowchart of the steps of Example 3 of a model training method provided in this application;

[0024] Figure 5 A flowchart of the steps of Embodiment 4 of a method for mental health detection provided by the present application;

[0025] Figure 6 This is a graph showing the different accuracy rates of the three language models involved in Example 5 of the present application on the validation set.

[0026] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0027] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0028] In order to better understand the above technical solution, exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0029] As an implementation solution, refer to Figure 1 , Figure 1 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present application.

[0030] like Figure 1As shown, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), a mouse, etc., and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0031] Those skilled in the art will understand that Figure 1 The terminal structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0032] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a model training program and / or a program for mental health detection.

[0033] exist Figure 1 In the terminal shown, the network interface 1004 is mainly used to connect to the background server and perform data communication with the background server; the processor 1001 can be used to call the training program of the model stored in the memory 1005 and perform the following operations:

[0034] Acquire a user psychological data set to be trained, and extract a feature representation sequence of each sample in the user psychological data set, wherein the user psychological data set includes data sets of multiple categories;

[0035] The improved BERT model is trained according to the feature representation sequence to obtain a pre-trained language model for mental health detection;

[0036] The loss function of the BERT model is improved to a cross-entropy weighted focal loss function, and the cross-entropy weighted focal loss function is:

[0037]

[0038] Wherein, G(y, p) represents the loss function value, i represents the index of the category, N represents the number of samples, and t represents the weight parameter. γ represents the hyperparameter, y irepresents the true probability that the sample belongs to the i-th category, p i represents the model prediction probability that the sample belongs to the i-th category, p k represents the output probability.

[0039] Further, the processor 1001 may call the training program of the model stored in the memory 1005, and further perform the following operations:

[0040] Obtain the user psychological data set to be trained;

[0041] The encoder of the BERT model is used to extract a feature representation sequence of each sample in the user psychological dataset.

[0042] Further, the processor 1001 may call the training program of the model stored in the memory 1005, and further perform the following operations:

[0043] Generate a corresponding tag embedding representation according to the tag information of the word itself in the sample;

[0044] According to the position of the word in the sentence, generate a corresponding position embedding representation;

[0045] Generate a corresponding fragment embedding representation according to the sentence or paragraph position of the word;

[0046] Integrate the token embedding representation, the position embedding representation and the fragment embedding representation to generate a feature representation of the word;

[0047] The feature representations of all the words in the sample are combined into the feature representation sequence.

[0048] Further, the processor 1001 may call the training program of the model stored in the memory 1005, and further perform the following operations:

[0049] Obtain a dataset for a specific task, wherein the dataset is annotated with input labels and output labels;

[0050] Training the pre-trained language model according to the data set of the specific task to obtain a model for mental health detection;

[0051] Among them, the model parameter optimization algorithm of the pre-trained language model is improved to the LORA algorithm.

[0052] exist Figure 1 In the terminal shown, the network interface 1004 is mainly used to connect to the background server and perform data communication with the background server; the processor 1001 can be used to call the program for mental health detection stored in the memory 1005 and perform the following operations:

[0053] Obtain user psychological data of the user to be tested;

[0054] The user psychological data of the user to be detected is input into the model for mental health detection obtained by training the model training method, and the mental health detection result of the user to be detected is obtained.

[0055] Based on the hardware architecture of the terminal based on the above-mentioned deep learning model technology, an embodiment of the model training method of the present invention is proposed.

[0056] Embodiment 1

[0057] Reference Figure 2 , Figure 2 This is a flowchart of the steps of the first embodiment of a model training method provided by the present application. In the first embodiment provided by the present application, the model training method includes the following steps S10 and S20:

[0058] Step S10: obtaining a user psychological data set to be trained, and extracting a feature representation sequence of each sample in the user psychological data set, wherein the user psychological data set includes data sets of multiple categories;

[0059] In some embodiments, the user psychological data set to be trained may be a psychological counseling conversation record, which refers to the content of multiple rounds of conversations with the user, including the user's emotional expression, problem statement, and the counselor's feedback and suggestions. Each sample in the user psychological data set may correspond to a psychological counseling conversation record of a user. The feature representation sequence refers to a sequence set composed of all feature representations of the sample.

[0060] In some embodiments, the user psychological data set to be trained includes data sets of multiple categories. As an example, the user psychological data set to be trained may include categories such as academic confusion and work and career worries, family problems and parent-child relationships, and interpersonal and emotional relationships. Further, in some embodiments, the data sets of each category can be further subdivided into data subsets of different categories to improve the model's refined analysis capabilities. As an example, the data sets of the academic confusion and work and career worries categories can be further subdivided into data subsets of anxiety, depression, pleasure, etc. according to emotional states, and / or subdivided into data subsets of social and work categories according to behavioral patterns. It should be noted that the embodiments of the present application do not limit the classification criteria of the user psychological data sets to be trained, and those skilled in the art can make adjustments based on the content and teachings disclosed in the embodiments of the present application.

[0061] In some embodiments, the user psychological data set to be trained can be obtained from the psychological counseling question and answer corpus of the official agency. The psychological counseling question and answer corpus records a wealth of psychological counseling conversation records, covering multiple rounds of conversation content and having detailed classification labels. Obtaining the user psychological data set to be trained from it can enable the model to deeply explore the specific psychological state information of the recorder from different dimensions, such as work pressure conditions, interpersonal relationship problems, and mood swings, etc., so as to accurately classify the recorder's emotions, identify the recorder's potential psychological problems and / or provide personalized mental health monitoring. As an example, the user psychological data set to be trained can be obtained from the "Psychological Counseling Question and Answer Corpus" released in April 2022. The psychological counseling records in the psychological counseling question and answer corpus are marked by professionals into multiple different categories. These categories reflect the various psychological problems and challenges that the recorder may face, which can help the model understand and analyze the recorder's psychological state more comprehensively.

[0062] In some embodiments, after obtaining the user data set to be trained from the psychological counseling question and answer corpus, a preprocessing operation may be performed to remove some meaningless data and symbolic data.

[0063] Step S20: training the improved BERT model according to the feature representation sequence to obtain a pre-trained language model for mental health detection; wherein the loss function of the BERT model is improved to a cross-entropy weighted focal loss function (CEWF), and the cross-entropy weighted focal loss function is:

[0064]

[0065] Wherein, G(y, p) represents the loss function value, i represents the index of the category, N represents the number of samples, and t represents the weight parameter. γ represents the hyperparameter, y i represents the true probability that the sample belongs to the i-th category, p i represents the model prediction probability that the sample belongs to the i-th category, p k represents the output probability.

[0066] In some embodiments, the BERT model is used as the basic model to improve the generalization ability of the pre-trained language model on various tasks, and the loss function of the BERT model is improved to a cross-entropy weighted focal loss function to optimize the deviation caused by the imbalance of input data, thereby improving the overall performance of the pre-trained language model, so that when the pre-trained language model is used for mental health testing, accurate test results can be obtained.

[0067] Next, the effect of the improved BERT model in the embodiment of the present application is described, but the embodiment of the present application is not limited.

[0068] The BERT model uses a Transformer architecture design, which includes a multi-head self-attention mechanism and a feedforward neural network layer, which enables it to effectively solve the problem of capturing long-distance dependencies in text and obtain and process global information of the input sequence. In addition, the BERT model also has a unique two-way context understanding capability, which means that it can consider the contextual content on both sides of the text at the same time. This shows that the BERT model can deeply understand the semantics of the text from a global and two-way perspective, thereby improving its performance in various natural language processing tasks.

[0069] The pre-training of the BERT model adopts an unsupervised learning method, which does not rely on labeled data and can use large-scale unlabeled text resources to learn language representations with wide applicability. Therefore, the embodiment of the present application adopts BERT as the basic model for training, which can improve the generalization ability of the pre-trained language model obtained by training on various tasks. In addition, during the pre-training process, the BERT model adopts the masked language model technology, that is, randomly masks some words in the input sequence, and then requires the model to predict the content of these masked words. In the above way, the model can be prompted to deeply understand the intrinsic semantic structure of the text. It is thus explained that the embodiment of the present application adopts the BERT model as the basic model for training, which can not only improve the generalization ability, but also significantly improve the depth and accuracy of the semantic understanding of the text, thereby generally improving the performance of the pre-trained language model obtained by training, so that when it is used for mental health testing, accurate test results can be obtained.

[0070] Furthermore, the loss function of the BERT model is the cross entropy classification loss function, which can be expressed as

[0071]

[0072] Among them, H(y, p) represents the loss function value of the BERT model, i represents the index of the category, N represents the number of samples, and y i represents the true probability that the sample belongs to the i-th category, p i represents the model prediction probability that the sample belongs to the i-th category, p k represents the output probability.

[0073] Output probability p k It can be expressed as:

[0074]

[0075] Where C represents the data classification feature vector, w represents the weight value to be trained, and b represents the bias value.

[0076] It can be found that the size of the loss function and the output probability p k Yes, the smaller the cross entropy loss function is, the smaller the difference between the model's prediction result and the true label is, and the better the model performance is. However, since cross entropy loss usually performs poorly on unbalanced data, during training, the loss of categories with a large number of data accounts for a large proportion of the total training loss, causing the model to be biased towards these categories. On the contrary, categories with less data have relatively low estimated probabilities due to their small sample size.

[0077] It should be noted that p i It is expressed as the probability that the model predicts the i-th category label based on the data. i can be equal to 1, 2, 3, etc., so there is a corresponding p 1 ,p 2 ,p 3 The probability values ​​of different labels. And p k It is the probability value of the category output by the model after predicting all categories, that is, after comparing the probability values, a category is output and the probability value of the category.

[0078] To address the above-mentioned defects, the embodiment of the present application improves the loss function of the BERT model to the cross-entropy weighted focal loss function described above, so that better training can still be achieved on data with less data volume, thereby reducing the impact of unbalanced data distribution on model training.

[0079] Specifically, based on the expression of the cross-entropy weighted focal loss function described above, the cross-entropy classification loss function introduces an adjustable hyperparameter γ , plus an adjustable weight parameter t to balance the impact of the original cross entropy loss function of the BERT model. When the weight parameter t=0, the improved BERT model loss function degenerates into a normal cross entropy loss function; when t>0, the improved BERT model loss function adjusts the weight according to the estimated probability value of the sample, so that the model pays more attention to those samples that are misclassified or those with poor classification results. At the same time, by adjusting the hyperparameters γ To control the model's attention to various data samples, so as to learn enough feature information from samples with a large amount of data, pay more attention to samples with a small number, and learn feature information, thereby reducing the loss value of the total data samples.

[0080] Based on the above discussion, it can be seen that the embodiment of the present application introduces a cross-entropy weighted focal loss function to give greater weight to misclassified samples during the training process, while making the model pay more attention to those categories that are less common in the data set. This two-pronged approach improves the performance of the model for these categories, thereby improving the training effect of the model to a certain extent and reducing the deviation caused by data imbalance.

[0081] In the technical solution provided in this embodiment, a user psychological data set to be trained is obtained, and a feature representation sequence of each sample in the user psychological data set is extracted, wherein the user psychological data set includes data sets of multiple categories, and then the improved BERT model is trained according to the feature representation sequence to obtain a pre-trained language model for mental health detection, wherein the BERT model loss function is improved to a cross-entropy weighted focal loss function. The embodiment of the present application uses the BERT model as the basic model and improves the BERT model loss function to a cross-entropy weighted focal loss function, which can not only improve the generalization ability of the pre-trained language model on various tasks, but also optimize the deviation caused by the imbalance of input data, so as to improve the performance of the pre-trained language model as a whole, so that when the pre-trained language model is used for mental health detection, accurate detection results can be obtained.

[0082] Embodiment 2

[0083] Reference Figure 3 , Figure 3 This is a flowchart of the steps of the second embodiment of a model training method provided by the present application. In the second embodiment, based on the above-mentioned first embodiment, the step S10 may include the following steps S11 and S12:

[0084] Step S11: Obtain the user psychological data set to be trained;

[0085] In this embodiment, the above embodiment 1 has given a specific implementation method for obtaining the user psychological data set to be trained, so the specific implementation method corresponding to step S11 in this embodiment can refer to embodiment 1, and no further details will be given here.

[0086] Step S12: Utilize the encoder of the BERT model to extract a feature representation sequence of each sample in the user psychological dataset.

[0087] In some embodiments, when the encoder of the BERT model processes input data, each word in the sample will go through the identification embedding process. Therefore, using the BERT model encoder to extract the feature representation sequence of the sample can significantly improve the model's understanding ability, especially in capturing long-distance dependencies, processing contextual information, and understanding the relationship between sentences, thereby further enhancing the model's performance in various natural language processing tasks and improving the performance of the pre-trained language model obtained through training.

[0088] Specifically, in some embodiments, step S12 may include: generating a corresponding tag embedding representation based on the tag information of the word itself in the sample. Generating a corresponding position embedding representation based on the position of the word in the sentence. Generating a corresponding fragment embedding representation based on the sentence or paragraph position of the word. Then integrating the tag embedding representation, the position embedding representation and the fragment embedding representation to generate a feature representation of the word. Finally, the feature representations of all words in the sample are combined into a feature representation sequence. It can be understood that the tag information of the word itself represents the semantic information of the word, and the tag embedding representation represents the semantic understanding of the word. The position embedding representation represents the specific position information of the word in the sentence. For example, if the word is the second word in the sentence, then the position embedding representation is the embedding representation corresponding to the second position in the sentence. The fragment embedding representation represents the sentence or paragraph position of the word. For example, if the word is in the third sentence, then the fragment embedding representation is the embedding representation corresponding to the third sentence.

[0089] In the technical solution provided in this embodiment, the encoder of the BERT model is used to extract the feature representation sequence of each sample in the user psychological data set to improve the model's understanding ability, thereby further enhancing the model's performance in various natural language processing tasks and improving the performance of the pre-trained language model obtained through training.

[0090] Embodiment 3

[0091] Reference Figure 4 , Figure 4 This is a flowchart of the steps of the third embodiment of a model training method provided by the present application. In the third embodiment, based on any of the above embodiments, after the step S20, the model training method may further include the following steps S30 and S40:

[0092] Step S30: obtaining a data set for a specific task, wherein the data set is annotated with input labels and output labels;

[0093] In some embodiments, the pre-trained language model obtained based on the above-mentioned embodiment one or embodiment two is trained by unsupervised learning of different categories of user psychological data, and what is learned is a general language representation ability, so that the pre-trained language model can capture the basic laws of language and has a wide range of language understanding capabilities. Therefore, before applying the pre-trained language model to the prediction of a specific task, the pre-trained language model needs to be trained again, and then applied to the prediction of the specific task. The data set for the specific task refers to a data set corresponding to a specific category, for example, it can be a task of psychological emotional tendency analysis and emotion classification. This embodiment does not limit the specific type of the task, and those skilled in the art can determine the specific type of the task according to the application scenario.

[0094] In some embodiments, the dataset for a specific task can also be obtained from the psychological counseling question and answer corpus of an official institution, and this embodiment does not specifically limit this. Unlike previous unsupervised learning, when training a dataset for a specific task, the pre-trained language model performs supervised learning on a dataset labeled with input labels and output labels, so as to be able to learn from the input data how to correctly predict the output label.

[0095] Step S40: training the pre-trained language model according to the data set of the specific task to obtain a model for mental health detection; wherein the model parameter optimization algorithm of the pre-trained language model is improved to a LORA (Low-Rank Adaptation) algorithm.

[0096] In some embodiments, the model parameter optimization algorithm of the pre-trained language model is used to fine-tune the model parameters during supervised learning training of a data set for a specific task, so that the trained model for mental health testing can predict accurate mental health testing results for the user psychological data corresponding to the specific task.

[0097] The model parameter optimization algorithm of the traditional BERT model is the Adam (Adaptive Moment Estimation) optimizer, which uses different learning rates on different parameters and dynamically adjusts the learning rate according to the first-order moment estimate and second-order moment estimate of the gradient of each parameter to accelerate the convergence speed and improve the model performance by adaptively adjusting the learning rate. However, the learning rate of Adam in the later stage of training is very small, which will affect the effective convergence. In addition, Adam may overfit the features that appeared in the early stage, and the features that appear in the later stage are difficult to correct the fitting effect of the early stage. Therefore, the embodiment of the present application improves the model parameter optimization algorithm of the pre-trained language model to the LORA algorithm, and uses the characteristic of the LoRA algorithm to express the model parameters through low-rank approximation during the training process, thereby constraining and reducing the dimension of the pre-trained language model parameters, thereby reducing the risk of overfitting of the pre-trained language model, and at the same time improving the generalization ability of the pre-trained language model on unseen data.

[0098] Specifically, in some embodiments, the LoRA algorithm represents the model parameters as the product of the original parameters and the low-rank matrix, and then adjusts the model parameters by adjusting the parameters of the low-rank matrix. During the supervised learning training of the pre-trained language model, LoRA simultaneously updates the model parameters and the parameters of the low-rank matrix by minimizing the loss function, so that the model can achieve better fitting effects on the training data and show stronger generalization ability on the test data, thereby improving the accuracy of the detection results output by the model.

[0099] In the technical solution provided in this embodiment, a data set of a characteristic task is obtained, wherein the data set is annotated with input labels and output labels, and then a pre-trained language model is trained according to the data set of the specific task to obtain a model for mental health detection, wherein the model parameter optimization algorithm of the pre-trained language model is improved to the LORA algorithm. The embodiment of the present application improves the model parameter optimization algorithm of the pre-trained language model to the LORA algorithm, which can improve the generalization ability of the trained model for mental health detection, thereby improving the accuracy of the detection results output by the model.

[0100] Embodiment 4

[0101] Reference Figure 5 , Figure 5 This is a flowchart of the steps of Embodiment 4 of a method for mental health detection provided by the present application. In Embodiment 4, based on any of the above embodiments, the method for mental health detection provided by the present application includes the following steps S50 and S60:

[0102] Step S50: obtaining user psychological data of the user to be detected;

[0103] In some embodiments, the user psychological data is the psychological data of the user to be detected that is recorded for a preset duration. The preset duration is not specifically limited in this embodiment. As an example, a software platform for recording user psychological data can be developed, so that the user can record his or her own psychological state information in real time through the software platform, thereby realizing the acquisition of user psychological data.

[0104] Step S50: Inputting the user psychological data of the user to be detected into the model for mental health detection obtained by training according to the model training method of any of the above embodiments, to obtain the mental health detection result of the user to be detected.

[0105] In some embodiments, the user psychological data of the user to be detected is input into the model for mental health detection obtained by training the training method of the model of any of the above embodiments, so as to obtain the psychological detection results of the user to be detected based on the model. In some embodiments, the mental health detection results may include, for example, the diagnosis results of the disease classification and / or the medical advice. The diagnosis result may include the user state and / or the cause of the user state. In some embodiments, the user state may be an overall description including the user's psychological state (cognition, emotion, will behavior change), physiological state (appetite, sleep, sexual function, weight), social interaction state (work, study and living conditions, efficiency and interpersonal communication). The cause of the user state may include the user's biological causes (disease, trauma and heredity), psychological causes (cognitive evaluation methods, personality deviation and negative emotional memory) and social causes (environmental influence and life events). In some embodiments, the model for mental health detection can further judge the nature and cause of the user's (visitor, helper) psychological problems from biological causes (disease, trauma and heredity), psychological causes (cognitive evaluation methods, personality deviation and negative emotional memory) and social causes (environmental influence and life events), and give medical advice.

[0106] It can be understood that the embodiment of the present application utilizes the user's psychological data recorded over a continuous period of time and adopts a model to perform psychological health testing on the user, which can not only obtain accurate test results, but also solve the lag and limitation problems of diagnosis by professionals.

[0107] In the technical solution provided in this embodiment, by obtaining the user psychological data of the user to be tested, the user psychological data of the user to be tested is input into a model for mental health testing obtained by training according to the training method of the model described in any of the previous embodiments, and the mental health test results of the user to be tested are obtained. Not only accurate test results can be obtained, but also the lag and limitations of diagnosis by professionals can be solved.

[0108] Embodiment 5

[0109] In this embodiment, the effect of the model proposed in the above embodiment is verified.

[0110] In this embodiment, the "Psychological Counseling Question and Answer Corpus" released in April 2022 is introduced. This is an important resource developed specifically for the use of artificial intelligence technology in the field of psychological counseling. As the first public QA data set in the field of psychological counseling, this corpus has more than 20,000 rich Chinese psychological counseling dialogue records, covering multiple rounds of dialogue content and detailed classification labels, allowing us to deeply explore the specific psychological state information of the recorder, such as work pressure, interpersonal relationship problems, and emotional fluctuations.

[0111] First, we preprocessed the data, extracted the required information, and performed category statistics on the entire data set. Since the data was manually labeled by professionals, the labels they made are considered to have a certain degree of credibility. After processing the data, we removed some meaningless data and symbolic data, and the statistical results are shown in Table 1:

[0112] Table 1. Preprocessed data table

[0113]

[0114] Next, the model proposed in the above embodiment is trained on this part of the data, and the effectiveness of the above training method is verified by recording the data and comparing it with other models. In order to compare the performance of other language models horizontally, the two language models XLNet and AT-LSTM with current powerful performance are selected for comparative experiments in this embodiment. They are also trained under this data set, and the performance of XLNet and AT-LSTM on the test set is recorded. The reason for selecting XLNet, AT-LSTM and the optimization model for comparison is their excellent performance in sentiment analysis and their wide application in different fields.

[0115] XLNet uses the Transformer-XL architecture, which enables it to use bidirectional contextual information for pre-training and fine-tuning, so that the model can better understand the context of the sentence and improve the model's performance on sentiment analysis tasks.

[0116] AT-LSTM introduces an attention mechanism, which can learn important information in the text and focus more on the parts that are helpful for sentiment classification, which improves the model's ability to extract key information and is also of great comparative significance for our experiment. In this example, the performance of XLNet, AT-LSTM and the optimized model in our experiment is compared to demonstrate the advantages and applicability of the optimized model, and further verify the pertinence and prominence of the optimized model in this experiment.

[0117] First, all data samples are divided into training set, validation set and test set with a division ratio of 6:2:2. LoRA is used for fine-tuning and training. The parameters are as shown in Table 2:

[0118] Table 2. Model parameter updates

[0119]

[0120] Set the parameters of XLNet and AT-LSTM, then use the pre-trained XLNet-Base model for training, and call the LSTM model containing Attention for training on the server. After training, record the accuracy, precision, loss value, and F1 score as evaluation indicators. Figure 6 The three language models shown have different accuracy graphs on the validation set.

[0121] It can be clearly seen that there are obvious differences in the accuracy of the three language models on the validation set. Among them, the accuracy of BERT+LoRA+CEWF has been maintained at a good level. It can be seen that compared with other language models, the optimized language model does have certain advantages in processing experimental tasks. As shown in Table 3 below, it is the performance data of the three language models on the test set:

[0122] Table 3. Performance of three language models on the test set

[0123]

[0124] It can be seen that XLNet performs well among the three language models, but its training time is relatively long. Although AT-LSTM performs well in the task, its performance is slightly inferior due to its lack of context capture ability. BERT+LoRA+CEWF combines BERT's powerful context capture ability with the optimization of LoRA and CEWF, and performs best with high accuracy and F1 score. In the same experimental environment and dataset, the BERT+LoRA+CEWF combined model performs well, outperforming XLNet and AT-LSTM. This shows that in the tasks handled by our experiment, the direction of the optimization strategy is correct, and the optimization model can better support the analysis of mental health monitoring.

[0125] In addition, to achieve the above-mentioned purpose, the present application also provides a terminal device, which includes: a memory, a processor, and a model training program and / or a program for mental health detection stored in the memory and run on the processor, wherein the model training program, when executed by the processor, implements the steps of the model training method of the terminal device as described above, and the program for mental health detection, when executed by the processor, implements the steps of the method for mental health detection of the terminal device as described above.

[0126] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which a model training program and / or a program for mental health detection is stored. When the model training program is executed by a processor, the steps of the model training method of the terminal device as described above are implemented. When the program for mental health detection is executed by a processor, the steps of the method for mental health detection of the terminal device as described above are implemented.

[0127] It should be noted that since the storage medium provided in the embodiment of the present application is the storage medium used to implement the method of the embodiment of the present application, based on the method introduced in the embodiment of the present application, the person skilled in the art can understand the specific structure and deformation of the storage medium, so it is not repeated here. All storage media used in the method of the embodiment of the present application belong to the scope of protection of this application.

[0128] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0129] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0130] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0131] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0132] It should be noted that in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0133] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0134] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A model training method, characterized in that: include: Acquire a user psychological data set to be trained, and extract a feature representation sequence of each sample in the user psychological data set, wherein the user psychological data set includes data sets of multiple categories; The improved BERT model is trained according to the feature representation sequence to obtain a pre-trained language model for mental health detection; The loss function of the BERT model is improved to a cross-entropy weighted focal loss function, and the cross-entropy weighted focal loss function is: ； Wherein, G(y, p) represents the loss function value, i represents the index of the category, N represents the number of samples, and t represents the weight parameter. γ represents the hyperparameter, y i represents the true probability that the sample belongs to the i-th category, p i represents the model prediction probability that the sample belongs to the i-th category, p k represents the output probability.

2. The training method of the model according to claim 1, characterized in that: The step of obtaining a user psychological data set to be trained and extracting a feature representation sequence of each sample in the user psychological data set includes: Obtain the user psychological data set to be trained; The encoder of the BERT model is used to extract a feature representation sequence of each sample in the user psychological dataset.

3. The training method of the model according to claim 2, characterized in that: The step of extracting a feature representation sequence of each sample in the user psychological dataset using the encoder of the BERT model comprises: Generate a corresponding tag embedding representation according to the tag information of the word itself in the sample; According to the position of the word in the sentence, generate a corresponding position embedding representation; Generate a corresponding fragment embedding representation according to the sentence or paragraph position of the word; Integrate the token embedding representation, the position embedding representation and the fragment embedding representation to generate a feature representation of the word; The feature representations of all the words in the sample are combined into the feature representation sequence.

4. The training method of the model according to claim 1, characterized in that: After the step of training the improved BERT model according to the feature representation sequence to obtain a pre-trained language model for mental health detection, the model training method further includes: Obtain a dataset for a specific task, wherein the dataset is annotated with input labels and output labels; Training the pre-trained language model according to the data set of the specific task to obtain a model for mental health detection; Among them, the model parameter optimization algorithm of the pre-trained language model is improved to the LORA algorithm.

5. A method for detecting mental health, characterized in that: include: Obtain user psychological data of the user to be tested; The user psychological data of the user to be detected is input into a model for mental health detection obtained by training according to the model training method described in any one of claims 1 to 4, to obtain the mental health detection result of the user to be detected.

6. The method for mental health detection according to claim 5, characterized in that: The user psychological data is the psychological data of the user to be detected that is recorded continuously for a preset period of time.

7. A terminal device, characterized in that: include: Memory; processor; as well as, A training program for a model for mental health detection of the terminal device stored in the memory and executable on the processor, and / or a program for mental health detection of the terminal device stored in the memory and executable on the processor; Wherein, when the training program of the model for mental health detection of the terminal device is executed by the processor, it implements the steps of the model training method as described in any one of claims 1 to 4, and when the program for mental health detection of the terminal device is executed by the processor, it implements the steps of the method for mental health detection as described in any one of claims 5 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a training program for a model for mental health detection and / or a program for mental health detection of a terminal device; wherein, when the training program for a model for mental health detection of the terminal device is executed by a processor, it implements the steps of the model training method as described in any one of claims 1 to 4, and when the program for mental health detection of the terminal device is executed by a processor, it implements the steps of the method for mental health detection as described in any one of claims 5 to 6.

Images