Electronic device for predicting risk levels for each mental disorder by using spoken text, and method therefor

Abstract

The present disclosure relates to an electronic device for predicting risk levels for each mental disorder by using spoken text, and a method therefor. Symptom onset prediction scores are output according to parameters for each mental disorder on the basis of tokens and a paragraph in which the tokens are arranged, so as to provide prediction results for a plurality of mental disorders on the basis of psychological counseling audio, and can be integrated and provided through a single UI.

Description

Electronic device and method for predicting risk level for each mental disorder using spoken text

[0001] The present disclosure relates to a device and method for predicting mental illness, and more particularly, to an electronic device and method for predicting a risk level for each mental illness using spoken text.

[0002] The number of people diagnosed with mental illness in Korea continues to increase, and the socioeconomic burden is increasing as productivity losses due to mental and behavioral disorders, as well as related crime and safety issues, are also on the rise.

[0003] Specifically, the number of people with mental illness increased by an average of 4.6% per year from 2.89 million in 2015 to 4.11 million in 2021, and the suicide rate per 100,000 people was 25.2 in 2022, the highest among OECD countries, and life satisfaction also ranked only 34th in OECD countries.

[0004] In particular, mental health problems such as depression, anxiety disorders, sleep disorders, and addiction are on the rise at a rapid pace.

[0005] However, while one in four adults in our country experiences mental illness at least once in their lifetime, only 11.5% of them receive professional help, making accessibility to professional services very low.

[0006] Additionally, treatment or counseling with mental health professionals is often undertaken after the mental illness has become severe, which has led to an increase in the socioeconomic costs of mental illness every year.

[0007] Therefore, there is a need to prevent and diagnose mental health problems in advance, understand one's emotional changes and psychological state, and manage one's mental health.

[0008] The purpose of the embodiment disclosed in the present disclosure is to provide an electronic device and method for predicting a risk level for each mental illness using spoken text, which outputs a symptom onset prediction score according to parameters for each mental illness based on tokens and paragraphs in which tokens are arranged to provide prediction results for multiple mental illnesses based on psychological counseling audio, and synthesizes and provides the scores in a single UI.

[0009] The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0010] In order to achieve the above-described technical task, an electronic device for predicting a risk level for each mental illness using a spoken text according to the present disclosure comprises: a first model for inputting psychological counseling audio and outputting a transcribed spoken text; a second model for inputting the spoken text and outputting a symptom onset prediction score according to at least one of a disease symptom, a risk factor, an improvement factor, and an intervention factor based on a token and a paragraph in which the token is arranged; a third model for classifying the symptom onset prediction score for each disease and outputting a corresponding risk level; a memory for storing at least one process for performing an operation for predicting a risk level for each mental illness; And at least one processor for performing the operation according to the process; wherein the at least one processor is configured to label at least one of symptoms, risk factors, improvement factors, and intervention factors for each disease for tokens in the spoken text as labeling data by the second model, and output a symptom onset prediction score for each disease by considering a weight for each labeling data according to the presence or absence or number of times of the labeling data, and the third model classifies a risk level according to the symptom onset prediction score for each disease and outputs the classified risk level for the entire disease in one UI.

[0011] In order to achieve the above-described technical task, a method for predicting a risk level for each mental illness using a spoken text according to the present disclosure comprises: a first model which receives psychological counseling audio as input and outputs a transcribed spoken text; a second model which receives the spoken text and outputs a symptom onset prediction score according to at least one of disease symptoms, risk factors, improvement factors, and intervention factors based on tokens and paragraphs in which the tokens are arranged; a third model which classifies the symptom onset prediction score for each disease and outputs a corresponding risk level; and a method for predicting a risk level for each mental illness using a spoken text of an electronic device including a memory and at least one processor, the method comprising: a step of labeling, by the second model, at least one of disease symptoms, risk factors, improvement factors, and intervention factors as labeling data for tokens in the spoken text; a step of outputting a symptom onset prediction score for each disease by considering a weight for each labeling data according to the presence or absence or number of the labeling data; And the third model may include a step of classifying the risk grade according to the symptom onset prediction score for each disease and outputting the classified risk grade for the entire disease as a single UI.

[0012] In addition, a computer program stored in a computer-readable recording medium for implementing the present disclosure may be further provided.

[0013] In addition, a computer-readable recording medium recording a computer program for implementing the present disclosure may be further provided.

[0014] According to the aforementioned problem solving means of the present disclosure, prediction results for multiple mental illnesses are provided based on psychological counseling audio, and mental illnesses are evaluated based on tokens and paragraphs in which tokens are arranged, thereby providing an effect of improving the accuracy of mental illness judgment.

[0015] In addition, the aforementioned problem solving means of the present disclosure provides the effect of improving user convenience by providing prediction results for multiple mental illnesses in a single UI, allowing the user to check the results at once.

[0016] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0017] FIG. 1 is a block diagram schematically illustrating the configuration of an electronic device for predicting a risk level for each mental disorder using spoken text according to one embodiment of the present disclosure.

[0018] FIG. 2 is a block diagram schematically illustrating a process-by-process model of an electronic device for predicting a risk level for each mental disorder using spoken text according to one embodiment of the present disclosure.

[0019] FIG. 3 is token and labeling data of spoken text according to one embodiment of the present disclosure.

[0020] FIG. 4 is a table showing labeling data of spoken text according to one embodiment of the present disclosure.

[0021] FIG. 5 is a diagram illustrating a disease prediction score output process of a second model of an electronic device that predicts a risk level for each mental illness using spoken text according to one embodiment of the present disclosure.

[0022] FIG. 6 and FIG. 7 are diagrams illustrating output result screens of an electronic device that predicts a risk level for each mental illness using spoken text according to one embodiment of the present disclosure.

[0023] FIG. 8 is a diagram illustrating a learning process that considers the relationship between paragraphs of an electronic device that predicts a risk level for each mental illness using spoken text according to one embodiment of the present disclosure.

[0024] FIG. 9 is a flow chart of a method for predicting a risk level for each mental disorder using spoken text according to one embodiment of the present disclosure.

[0025] Throughout this disclosure, the same reference numerals denote the same components. This disclosure does not describe all elements of the embodiments, and any content that is common in the technical field to which this disclosure pertains or that overlaps between embodiments is omitted. The terms "part, module, element, block" used in the specification may be implemented in software or hardware, and depending on the embodiments, multiple "parts, modules, elements, blocks" may be implemented as a single component, or a single "part, module, element, block" may include multiple components.

[0026] Throughout the specification, when a part is said to be "connected" to another part, this includes not only direct connection but also indirect connection, and indirect connection includes connection via a wireless communication network.

[0027] Additionally, when a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0028] Throughout the specification, when we say that an element is "on" another element, this includes not only cases where the element is in contact with the other element, but also cases where another element exists between the two elements.

[0029] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0030] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0031] The identification codes for each step are used for convenience of explanation and do not describe the order of each step. Each step may be performed in a different order than specified unless the context clearly indicates a specific order.

[0032] The operating principle and embodiments of the present disclosure are described below with reference to the attached drawings.

[0033] As used herein, the term "device according to the present disclosure" encompasses a variety of devices capable of performing computational processing and providing results to a user. For example, the device according to the present disclosure may include a computer, a server device, and a portable terminal, or may be any one of them.

[0034] Here, the computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

[0035] The above server device is a server that processes information by communicating with an external device, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0036] The above portable terminal may include, for example, a wireless communication device that ensures portability and mobility, and may include all kinds of handheld-based wireless communication devices such as a PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminal, a smart phone, and a wearable device such as a watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses, or a head-mounted device (HMD).

[0037] The artificial intelligence-related functions according to the present disclosure are operated via a processor and memory. The processor may be comprised of one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a DSP (Digital Signal Processor), a graphics-only processor such as a GPU or a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU. One or more processors control the processing of input data according to predefined operating rules or artificial intelligence models stored in memory. Alternatively, if one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0038] The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that the basic artificial intelligence model is learned by a learning algorithm using a plurality of learning data, thereby creating a predefined operation rules or artificial intelligence model set to perform a desired characteristic (or purpose). This learning may be performed on the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0039] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values, and performs neural network operations through operations between the operation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. The artificial neural network may include a deep neural network (DNN), and examples thereof include, but are not limited to, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or deep Q-networks.

[0040] According to an exemplary embodiment of the present disclosure, a processor can implement artificial intelligence. Artificial intelligence refers to a machine learning method based on an artificial neural network that imitates human neurons (biological neurons) to enable machines to learn. Artificial intelligence methodologies can be categorized into supervised learning, in which input data and output data are provided together as training data depending on the learning method, so that the solution (output data) to the problem (input data) is determined; unsupervised learning, in which only input data is provided without output data, so that the solution (output data) to the problem (input data) is not determined; and reinforcement learning, in which a reward (Reward) is provided from an external environment whenever an action (Action) is taken in the current state (State), and learning is performed in a direction to maximize this reward. In addition, artificial intelligence methodologies can be categorized according to the architecture of the learning model. The architectures of widely used deep learning technologies can be categorized into convolutional neural networks (CNNs), recurrent neural networks (RNNs), transformers, and generative adversarial networks (GANs).

[0041] The present device and system may include an artificial intelligence model. The artificial intelligence model may be a single artificial intelligence model or may be implemented as multiple artificial intelligence models. The artificial intelligence model may be composed of a neural network (or artificial neural network) and may include statistical learning algorithms that mimic biological neurons in machine learning and cognitive science. A neural network may refer to a model in general that has problem-solving capabilities by changing the binding strength of synapses through learning, formed by artificial neurons (nodes) that form a network by combining synapses. The neurons of the neural network may include a combination of weights or biases. The neural network may include one or more layers composed of one or more neurons or nodes. For example, the device may include an input layer, a hidden layer, and an output layer. The neural network constituting the device can infer a desired result (output) from an arbitrary input (input) by changing the weights of neurons through learning.

[0042] The processor can create a neural network, train (or learn) a neural network, perform a calculation based on received input data, generate an information signal based on the calculation result, or retrain the neural network. The models of the neural network can include various types of models such as CNN (Convolution Neural Network) such as GoogleNet, AlexNet, VGG Network, R-CNN (Region with Convolution Neural Network), RPN (Region Proposal Network), RNN (Recurrent Neural Network), S-DNN (Stacking-based deep Neural Network), S-SDNN (State-Space Dynamic Neural Network), Deconvolution Network, DBN (Deep Belief Network), RBM (Restrcted Boltzman Machine), Fully Convolutional Network, LSTM (Long Short-Term Memory) Network, Classification Network, etc., but are not limited thereto. The processor can include one or more processors for performing calculations according to the models of the neural network. For example, the neural network can be a deep neural network. It may include a deep neural network.

[0043] Neural networks include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Function), DFF (Deep Feed Forward), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto) Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), Generative Adversarial Network (GAN), Liquid State Machine (LSM), Extreme Learning Machine (ELM), It will be understood by those skilled in the art that any neural network may be included, including but not limited to ESN (Echo State Network), DRN (Deep Residual Network), DNC (Differentiable Neural Computer), NTM (Neural Turning Machine), CN (Capsule Network), KN (Kohonen Network), and AN (Attention Network).

[0044] According to an exemplary embodiment of the present disclosure, the processor may be configured to perform a process for generating a CNN (Convolution Neural Network) such as GoogleNet, AlexNet, VGG Network, Region with Convolution Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restrcted Boltzman Machine (RBM), Fully Convolutional Network, Long Short-Term Memory (LSTM) Network, Classification Network, Generative Modeling, eXplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT, SP-BERT, MRC / QA for natural language processing, Text Analysis, Dialog System, GPT-3, GPT-4, Visual Analytics for vision processing, Visual Understanding, Video Synthesis, ResNet for data intelligence, Anomaly Detection, Prediction, Time-Series Forecasting, Various artificial intelligence structures and algorithms, including optimization, recommendation, and data creation, can be utilized, but are not limited thereto. Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0045] FIG. 1 is a block diagram briefly illustrating the configuration of an electronic device (100) that predicts a risk level for each mental illness using spoken text according to one embodiment of the present disclosure.

[0046] Referring to FIG. 1, an electronic device (100) according to the present disclosure may include an input / output module (110), a communication module (120), a memory (130), and a processor (140). Hereinafter, it is assumed that the electronic device (100) is an electronic device for predicting a risk level for each mental illness, and that the method thereof is implemented through the electronic device (100).

[0047] The input / output module (110) may be various interfaces or connection ports that receive user input or output information to the user. The input / output module (110) may be divided into an input module and an output module.

[0048] The input module receives user input from the user. The input module is for inputting video information (or signals), audio information (or signals), data, or information input from the user, and may include at least one camera, at least one microphone, and at least one user input unit. Voice data or image data collected by the input unit may be analyzed and processed into user control commands.

[0049] User input can take many forms, including key input, touch input, and voice input. Examples of input modules that can receive such user input include, in addition to traditional keypads, keyboards, and mice, touch sensors that detect the user's touch, microphones that receive voice signals, cameras that recognize gestures through image recognition, proximity sensors such as light sensors or infrared sensors that detect user approach, motion sensors that recognize user movements such as acceleration sensors or gyro sensors, and a comprehensive concept that includes all types of input means that detect or receive various types of user input.

[0050] Here, the touch sensor may be implemented as a piezoelectric or electrostatic touch sensor that detects touch through a touch panel or touch film attached to the display panel, or an optical touch sensor that detects touch using an optical method. In addition, the input module may be implemented in the form of an input interface (such as a USB port or PS / 2 port) that connects an external input device that receives user input instead of a device that detects user input on its own.

[0051] An output module can output various types of information and provide them to the user. An output module is a comprehensive concept that includes a display that outputs images, a speaker (and / or an amplifier connected to it) that outputs sound, a haptic device that generates vibrations, and various other forms of output means. Additionally, an output module can be implemented in the form of a port-type output interface that connects the individual output means described above.

[0052] For example, an output module in the form of a display can display text, still images, and moving images. The display is a concept that broadly refers to an image display device that includes various forms of devices that can perform image output functions, such as a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a flat panel display (FPD), a transparent display, a curved display, a flexible display, a 3D display, a holographic display, a projector, and other devices. Such a display may be in the form of a touch display that is configured integrally with a touch sensor of the input module.

[0053] In other words, the input / output module (110) can receive user input or provide output to the user based on a user interface.

[0054] The communication module (120) can communicate with an external device. Accordingly, the device can transmit and receive information with the external device through the communication module. For example, the device can communicate with the external device using the communication module so that information stored and generated within the electric vehicle charging management system can be shared. The communication module (120) can include, for example, at least one of a wired communication module, a wireless communication module, a short-range communication module, and a location information module.

[0055] Here, communication, i.e., data transmission and reception, can be performed wired or wirelessly. To this end, the communication module may be configured as a wired communication module that connects to the Internet, etc., via a LAN (Local Area Network), a mobile communication module that connects to a mobile communication network via a mobile communication base station to transmit and receive data, a short-range communication module that uses a WLAN (Wireless Local Area Network) series communication method such as Wi-Fi or a WPAN (Wireless Personal Area Network) series communication method such as Bluetooth or Zigbee, a satellite communication module that uses a GNSS (Global Navigation Satellite System) such as a GPS (Global Positioning System), or a combination thereof. The wireless communication technology used for communication may include NB-IoT (Narrowband Internet of Things) for low-power communication. At this time, for example, NB-IoT technology may be an example of LPWAN (Low Power Wide Area Network) technology and may be implemented with standards such as LTE Cat (category) NB1 and / or LTE Cat NB2, and is not limited to the above-described names. Additionally or alternatively, wireless communication technology implemented in wireless devices according to various embodiments may perform communication based on LTE-M technology. At this time, for example, LTE-M technology may be an example of LPWAN technology and may be called by various names such as eMTC (enhanced Machine Type Communication).For example, LTE-M technology can be implemented by at least one of various standards such as 1) LTE CAT 0, 2) LTE Cat M1, 3) LTE Cat M2, 4) LTE non-BL (non-Bandwidth Limited), 5) LTE-MTC, 6) LTE Machine Type Communication, and / or 7) LTE M, and is not limited to the above-described names. Additionally or alternatively, wireless communication technologies implemented in wireless devices according to various embodiments can include at least one of ZigBee, Bluetooth, and Low Power Wide Area Network (LPWAN) that take low-power communication into account, and are not limited to the above-described names. For example, ZigBee technology can create personal area networks (PANs) related to small / low-power digital communication based on various standards such as IEEE 802.15.4, and can be called by various names.

[0056] The memory (130) can store various types of information. The memory can store data temporarily or semi-permanently. For example, the memory can store operating programs (OS: Operating System) for operating the first device and / or the second device, data for hosting a website, a program for generating Braille, or data related to an application (e.g., a web application). Furthermore, the memory can store modules in the form of computer code, as described above.

[0057] Examples of memory (130) may include a hard disk drive (HDD), a solid state drive (SSD), flash memory, read-only memory (ROM), random access memory (RAM), etc. Such memory may be provided as a built-in type or a removable type.

[0058] The processor (140) controls the overall operation of the electronic device (100). To this end, the processor (140) can perform calculations and processing of various types of information and control the operation of components of the first device and / or the second device.

[0059] The processor (140) may be implemented as a computer or similar device according to hardware, software, or a combination thereof. In terms of hardware, the processor (140) may be provided in the form of an electronic circuit that processes electrical signals to perform control functions, and in terms of software, the processor (140) may be provided in the form of a program that drives the hardware processor. Meanwhile, unless otherwise specified in the following description, the operations of the first device and / or the second device may be interpreted as being performed under the control of the processor (140). That is, the modules may be interpreted as controlling the processor (140) to cause the first device and / or the second device to perform the following operations.

[0060] The processor (140) may be implemented as a memory storing data for an algorithm for controlling the operation of components within the device or a program reproducing the algorithm, and at least one subprocessor (not shown) that performs the aforementioned operations using the data stored in the memory. In this case, the memory and the processor may be implemented as separate chips. Alternatively, the memory and the processor may be implemented as a single chip.

[0061] In addition, the processor (140) can control any one or a combination of the components described above to implement various embodiments according to the present disclosure described in the drawings below on the device.

[0062] FIG. 2 is a block diagram illustrating a process model of an electronic device (100) that predicts a risk level for each mental illness using spoken text according to an embodiment of the present disclosure, FIGS. 3 and 4 are tables illustrating a labeling process and labeling data of spoken text according to an embodiment of the present disclosure, FIG. 5 is a diagram illustrating a disease prediction score output process of a second model (220) of an electronic device (100) according to an embodiment of the present disclosure, FIGS. 6 and 7 are diagrams illustrating an output result screen of an electronic device (100) according to an embodiment of the present disclosure, and FIG. 8 is a diagram illustrating a learning process that considers a relationship between paragraphs of an electronic device (100) according to an embodiment of the present disclosure.

[0063] As illustrated in FIG. 2, an electronic device (100) for predicting a risk level for each mental illness using a spoken text according to the present disclosure may include a first model (210) that receives psychological counseling audio and outputs a transcribed spoken text, a second model (220) that receives the spoken text and outputs a symptom onset prediction score according to at least one of disease symptoms, risk factors, improvement factors, and intervention factors based on tokens and paragraphs in which the tokens are arranged, a third model (230) that classifies the symptom onset prediction score for each disease and outputs a corresponding risk level, a memory (130) in which at least one process for performing an operation of predicting a risk level for each mental illness is stored, and at least one processor (140) that performs the operation according to the process.

[0064] At least one processor (140) according to one embodiment of the present disclosure may be configured such that the second model (220) labels at least one of a symptom, a risk factor, an improvement factor, and an intervention factor for each disease as labeling data for tokens in the spoken text, and outputs a symptom onset prediction score for each disease by considering a weight for each labeling data according to the presence or absence or number of times of the labeling data, and the third model (230) classifies a risk level according to the symptom onset prediction score for each disease and outputs the classified risk level for the entire disease as one UI.

[0065] As shown in FIG. 2, the first model (210), the second model (220), and the third model (230) can be arranged sequentially.

[0066] The first model (210) can preprocess psychological counseling audio into spoken text to enable data processing.

[0067] As an example, the first model (210) may be a STT (Speech-to-Text) model.

[0068] The second model (220) can perform score prediction after generating and labeling tokens for each paragraph. Tokens can be generated by considering paragraphs and core keywords for the transcribed spoken text, and data matching the tokens can be labeled.

[0069] As an example, the second model (220) may be a BERT-based model that performs natural language processing.

[0070] As an example, it may include a BERT-based model that inputs token and label data corresponding to each disease and outputs a score for each disease.

[0071] Additionally, based on labeled data, a weighted score can be calculated to predict symptom onset for each disease. This score can be in the form of a probability distribution representing the probability of symptom occurrence for each disease, or a classification score that calculates the probability of symptom occurrence for each disease.

[0072] The third model (230) can predict scores for all diseases. The disease-specific scores output from each BERT-based model of the second model (220) can be combined and output as a single UI that reflects the risk of each disease.

[0073] An electronic device (100) according to one embodiment of the present disclosure can predict the risk of disease onset based on psychological counseling audio and speech text (21) transcribed from the psychological counseling audio.

[0074] As shown in Fig. 3, tokens can be generated by tokenizing main keywords in the transcribed speech text (21) of the audio of psychological counseling conducted between the counselor and the client, taking paragraphs into consideration.

[0075] For example, referring to Figure 3, 'difficult and depressed' (21a) may be a token corresponding to a depressive symptom, and 'a feeling of fear as if I were going to die' (21b) may be a token corresponding to an anxiety symptom.

[0076] Tokens can be generated by considering words in the spoken text and paragraphs containing said words, and labeling data such as 'depressive symptoms' or 'anxiety symptoms' can be mapped to said tokens.

[0077] The above words may be words stored in a preset database. Even paragraphs containing the same words may be mapped to different labeling data depending on the context and intent of the paragraph.

[0078] The words stored in the above database may be words established by medical domain experts or words built into a specialized medical terminology dictionary.

[0079] In addition, labeling data can be mapped by extracting not only specific words of the spoken text (21), but also linguistic elements such as tone, speaking speed, and voice vibration from the psychological counseling audio before the spoken text (21) is transcribed.

[0080] The second model (220) can obtain speech text (21) from the first model (210) and psychological counseling audio from the input / output module (110) and output a symptom onset prediction score by comprehensively considering tokens, labeling data, and linguistic elements.

[0081] Accordingly, according to one embodiment of the present disclosure, the system further includes an input / output module (110) for collecting a user's psychological counseling audio, and the at least one processor (140) may be configured to allow the second model (220) to receive psychological counseling audio and speech text from the input / output module and the first model, cross-fuse them into multi-modal data, and output a symptom onset prediction score for each disease based on the speech speed, tone, and token of the multi-modal data.

[0082] Meanwhile, referring to FIG. 4, labeling data can be matched to tokens based on symptoms and factors according to the classification criteria (400) of FIG. 4.

[0083] As an example, labeling data can be categorized into key symptoms, risk factors, improvement factors, and intervention factors.

[0084] For example, as shown in Figure 4, it can be classified into depressive symptoms, depression risk factors, anxiety symptoms, anxiety risk factors, addiction symptoms, addiction risk factors, improvement factors, and counselor intervention factors.

[0085] As an example, symptoms of depression can be categorized as depressed mood, decreased interest, weight / appetite changes, sleep problems, psychomotor changes, fatigue, listlessness, guilt, impaired thinking, and suicidal thoughts.

[0086] Risk factors for depression can be categorized as family history, underlying medical conditions, past mental illness, traumatic experiences, derogatory beliefs, negative self-image, lack of coping strategies, low self-control, stressful events, and lack of social support.

[0087] Anxiety symptoms can be categorized into feelings of anxiety and fear, difficulty controlling anxiety, physical symptoms, fatigue, difficulty concentrating, irritability, sleep problems, feelings of unreality, feelings of loss of control, and social situations.

[0088] Anxiety risk factors can be classified using the same criteria as depression risk factors, or they can be classified by including additional risk factors specific to anxiety.

[0089] Addiction symptoms and risk factors can also be classified according to the same criteria.

[0090] Additionally, improvement factors can be classified as symptom improvement factors into cognitive improvement, emotional improvement, and behavioral improvement.

[0091] In addition, intervention factors can be classified as counselor intervention factors, including empathy / supportive response, clarification / restatement, cognitive restructuring, information provision, behavioral intervention, task assignment, emotional regulation, goal / plan establishment, process feedback, and coping skills.

[0092] According to the above-described classification, labeling data can be labeled to tokens, and different weights can be applied to each labeling data as a high-risk symptom or factor, a medium-risk symptom or factor, or a low-risk symptom or factor.

[0093] For example, anxiety and fear could be considered high-risk symptoms and given a higher weight in the symptom onset prediction score. Alternatively, family history could be considered a high-risk anxiety risk factor and given a higher weight in the symptom onset prediction score.

[0094] Conversely, fatigue in depressive symptoms could be considered a medium-risk symptom and reflected by applying a medium weight to the symptom onset prediction score. Alternatively, negative self-image could be considered a low-risk factor for depression and reflected by applying a low weight to the symptom onset prediction score.

[0095] In other words, if a word is identical or similar to a core keyword stored in the database and maps to the classification criteria of the labeling data in the context of the paragraph, the corresponding labeling data can be mapped to a token.

[0096] Accordingly, a second model (220) according to one embodiment of the present disclosure may include a tokenizer module that extracts words corresponding to each disease by considering the context or paragraph in the spoken text and generates tokens, a labeling module that maps labeling data for each disease to the tokens, and a score prediction module that receives a token and labeling data corresponding to one disease among all tokens and outputs a symptom expression prediction score for the one disease.

[0097] As an example, the score prediction module may include a plurality of Ko-Bert models (221, 222, 223) that input tokens and labeling data corresponding to the entire disease and output a symptom onset probability distribution for each corresponding disease or a risk score that applies the weight of the labeling data to the symptom onset probability distribution.

[0098] As shown in Fig. 5, the speech text (21) to which token and labeling data are mapped can be input to the Ko-Bert model (221, 222, 223) for the entire disease without distinguishing by disease.

[0099] The above plurality of Ko-Bert models (221, 222, 223) can be arranged in parallel to receive token and labeling data corresponding to the same overall disease.

[0100] Each Ko-Bert model (221, 222, 223) can receive a speech text (21) mapped with token and label data for the entire disease and extract and process token and label data for the disease corresponding to each Ko-Bert model (221, 222, 223).

[0101] To avoid wasting resources by repeatedly inputting some common criteria for labeling data classification for each disease, a single full disease token mill labeling data can be input.

[0102] As an example, by arranging the depression Ko-Bert model (221), the anxiety Ko-Bert model (222), and the addiction Ko-Bert model (223), it is possible to derive a prediction score for each depression subfactor, a prediction score for each anxiety subfactor, and a prediction score for each addiction subfactor for each model.

[0103] Each Ko-Bert model (221, 222, 223) can output a risk probability distribution (31) for each disease or a risk score (32) for each disease.

[0104] As an example, a probability distribution for each disease can be derived based on Softmax. Depending on the labeled data for each disease, the probability of high, medium, or low risk can be output.

[0105] For example, using token and label data related to depression, the probability can be output as high, medium, low, or none, or using token and label data related to anxiety, the probability can be output as high, medium, or low, or using token and label data related to addiction, the probability can be output as high, medium, or low, or none.

[0106] Alternatively, a classification score that can classify risk levels can be calculated by applying different weights to the probability distribution, such as high, medium, low, and none.

[0107] For example, a classification score can be calculated by multiplying the probability and weight by 3 for high, 2 for medium, 1 for low, and 0 for none. Referring to Figure 5, a classification score can be calculated for a probability distribution of depression risk output with a probability of 0.6 for high, 0.2 for medium, 0.1 for low, and 0.1 for none, resulting in a depression score of 2.3.

[0108] Accordingly, the second model (220) can output a probability distribution or a classification score with different weights applied to the probability distribution.

[0109] Meanwhile, the third model (230) can output a single, aggregated score for each disease. Referring to Fig. 5, by passing the probability distribution or classification score through a threshold for each risk classification, the probability distribution or classification score can be classified into a risk grade corresponding to the corresponding probability distribution or classification score.

[0110] For example, a depression score of 2.3 would correspond to a mid-level risk rating and be output as "Medium." The corresponding risk ratings for anxiety and addiction could also be output.

[0111] The risk ratings outputted can be outputted to a single UI (400). A screen implemented as a single UI (400) can be outputted to the input / output module (110) so that each disease can be checked at once on a single screen without having to check on a separate screen.

[0112] Referring to Figure 6, it can be confirmed that the risk levels for each disease are comprehensively displayed at once within a single UI (400). At this time, the single UI (400) may be a table in which the names of each disease and the risk levels of the diseases are displayed correspondingly.

[0113] Alternatively, as illustrated in FIG. 7, it may be a graph in which the risk levels for each disease are comprehensively displayed at once within one UI (401).

[0114] For each disease, the symptoms that occur most frequently and those that occur rarely can be displayed in a diagram, and the graph can be displayed to show the disease with the highest risk among all diseases.

[0115] Additionally, the overall results and detailed results for each disease can be displayed together.

[0116] As an example, the third model (230) may be a Fully Connected Deep Neural Network (FC-DNN) model.

[0117] Meanwhile, at least one processor (140) according to one embodiment of the present disclosure is configured to enable the second model (220) to perform learning using learning data, and the learning data may include paragraphs containing tokens corresponding to symptoms, risk factors, improvement factors, and intervention factors for each disease, and labeling data of the tokens.

[0118] In addition, as an embodiment, it may further include a user interface-based input / output module (110) that receives user input and a database (not shown) that stores the learning data.

[0119] At least one processor (140) according to one embodiment of the present disclosure is configured to receive the token and the labeling data of the token from the input / output module (110), and may be configured to store the token, a paragraph including the token corresponding to the symptom, risk factor, improvement factor, and intervention factor of the disease, and the labeling data of the token in the database.

[0120] As one embodiment, at least one processor (140) can train the second model (220) to perform labeling based on context or paragraphs by considering not only tokens but also relationships between sentences or paragraphs in which tokens are placed.

[0121] As illustrated in FIG. 8, the second model (222) may include each transformer (222) that receives a text containing 1 token, 2 tokens, and n tokens within the text.

[0122] Each transformer (222) can extract features within the input text.

[0123] The extracted features can be input into a dense layer to extract tokens, which are core keywords, and label the labeling data of the tokens.

[0124] For example, a transformer (222) may receive text containing a first token as input, extract feature F1, and input F1 into a dense layer to extract the first token in the text and classify labeling data corresponding to the first token.

[0125] As an example, feature F1 extracted from text containing a first token can be input as feature F2 output by a transformer (222) that receives text containing a second token (S810).

[0126] This allows outputting a feature that compensates for the feature F2 of the second token by considering the context or paragraph relationship between the text containing the second token and the text containing the first token.

[0127] The corrected feature can be input into a dense layer to extract a second token in the text and classify label data corresponding to the second token.

[0128] Additionally, feature F2 extracted from text containing the second token can be input as feature F3 output by a transformer (222) that inputs text containing the third token (S820).

[0129] This allows outputting a feature that compensates for the feature F3 of the third token by considering the context or paragraph relationship between the text containing the third token and the text containing the second token.

[0130] Each transformer (222) that performs classification to determine disease symptoms, risk factors, improvement factors and intervention factors by considering the context can receive the features of the previous paragraph as input.

[0131] When inputting the features of the previous paragraph into the features of the current paragraph, the features of the current paragraph can be added to the features of the previous paragraph calculated using the global mean pool.

[0132] Meanwhile, if the output result of the second model (220) deviates from the standard value or an overfitting problem occurs in the inference performance, an action to modify the weights within the second model (220) can be performed.

[0133] As an example, if the first token extraction and labeling data is unsuitable, the weights may be modified to increase the reward for the transformer (222) performing the first token extraction (S830).

[0134] As an example, if the second token extraction and labeling data is unsuitable, the weights may be modified to increase the reward for the transformer (222) that extracts the second token (S840).

[0135] As an example, if the nth token extraction and labeling data is unsuitable, the weights can be modified to increase the reward for the transformer (222) that extracts the nth token (S850).

[0136] Therefore, the weights for each model outputting each token can be modified to enable accurate token and label data classification.

[0137] FIG. 9 is a flow chart of a method for predicting a risk level for each mental disorder using spoken text according to one embodiment of the present disclosure.

[0138] The above method may be a method for predicting a risk level for each mental illness using the speech text of an electronic device including a first model (210) that inputs psychological counseling audio and outputs a transcribed speech text, a second model (220) that inputs the speech text and outputs a symptom onset prediction score according to at least one of disease symptoms, risk factors, improvement factors, and intervention factors based on tokens and paragraphs in which the tokens are arranged, a third model (230) that classifies the symptom onset prediction score for each disease and outputs a corresponding risk level, and a memory (130) and at least one processor (140).

[0139] In the above method, the second model (220) may include a step (S910) of labeling at least one of a symptom, a risk factor, an improvement factor, and an intervention factor for each disease as labeling data for tokens in the utterance text, a step (S920) of outputting a symptom onset prediction score for each disease by considering a weight for each labeling data according to the presence or absence or number of times of the labeling data, and a step (S930) of the third model (230) of classifying a risk level according to the symptom onset prediction score for each disease and outputting the classified risk level for all diseases as one UI (400).

[0140] In step (S920), the second model (220) tokenizes words corresponding to disease-specific symptom factors and risk factors, disease improvement factors, and counselor intervention factors through a tokenizer module, and maps each matching factor to labeling data.

[0141] In step (S930), one UI (400) outputs at least the risk of occurrence for each disease on the interface of the input / output module (110) without moving the screen, and additionally, a detailed review or comprehensive review can be displayed.

[0142] Any content that overlaps with the above will be omitted for the sake of brevity of the specification.

[0143] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium storing computer-executable instructions. The instructions may be stored in the form of program code, and when executed by a processor, may generate program modules to perform the operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0144] Computer-readable storage media include all types of storage media that store instructions that can be deciphered by a computer. Examples include read-only memory (ROM), random access memory (RAM), magnetic tape, magnetic disks, flash memory, and optical data storage devices.

[0145] The disclosed embodiments have been described with reference to the attached drawings as described above. Those skilled in the art will understand that the present disclosure can be implemented in forms other than the disclosed embodiments without altering the technical spirit or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be construed as limiting.

Claims

1. A first model that inputs psychological counseling audio and outputs transcribed speech text; A second model that inputs the above-mentioned speech text and outputs a symptom onset prediction score according to at least one of disease symptoms, risk factors, improvement factors, and intervention factors based on tokens and paragraphs in which the tokens are placed; A third model that classifies the symptom onset prediction score for each disease and outputs a corresponding risk grade; A memory storing at least one process for performing an action to predict a risk rating for a mental disorder; and At least one processor for performing the above operation according to the above process; At least one processor, The second model labels at least one of disease-specific symptoms, risk factors, improvement factors, and intervention factors for tokens in the utterance text as labeling data, and outputs a symptom expression prediction score for each disease by considering a weight for each labeling data according to the presence or absence or number of times of the labeling data. The above third model is configured to classify the risk grade according to the symptom onset prediction score for each disease and output the classified risk grade for all diseases in one UI. An electronic device that predicts risk ratings for mental disorders using spoken text.

2. In paragraph 1, It further includes an input / output module for collecting the user's psychological counseling audio, At least one processor, the second model, Receive psychological counseling audio and speech text from the above input / output module and the first model and cross-fuse them into multi-modal data, It is configured to output a symptom onset prediction score for each disease based on the speech rate, tone, and token of the above multi-modal data. An electronic device that predicts risk ratings for mental disorders using spoken text.

3. In paragraph 1, The second model is, A tokenizer module that extracts words corresponding to each disease by considering the context or paragraph in the above-mentioned speech text and generates tokens; A labeling module that maps labeling data for each disease to the above token; and A score prediction module that inputs token and label data corresponding to one disease among all tokens and outputs a symptom onset prediction score for the one disease; An electronic device that predicts risk ratings for mental disorders using spoken text.

4. In paragraph 3, The above score prediction module includes multiple Ko-Bert models that input token and label data corresponding to the entire disease and output a symptom onset probability distribution for each corresponding disease or a risk score that applies the weight of the label data to the symptom onset probability distribution. An electronic device that predicts risk ratings for mental disorders using spoken text.

5. In paragraph 4, The above multiple Ko-Bert models are characterized in that they are arranged in parallel to receive token and label data corresponding to the same overall disease. An electronic device that predicts risk ratings for mental disorders using spoken text.

6. In paragraph 1, The third model is characterized by being a Fully Connected Deep Neural Network (FC-DNN) model. An electronic device that predicts risk ratings for mental disorders using spoken text.

7. In paragraph 1, At least one processor, The above second model is configured to perform learning using learning data, The above learning data includes paragraphs containing tokens corresponding to symptoms, risk factors, improvement factors, and intervention factors for each disease, and labeling data of the tokens. An electronic device that predicts risk ratings for mental disorders using spoken text.

8. In paragraph 7, A user interface-based input / output module that receives user input, Further comprising a database storing the above learning data, At least one processor, It is configured to receive the token and the labeling data of the token from the input / output module, The above token, a paragraph containing tokens corresponding to the symptoms, risk factors, improvement factors and intervention factors of the above disease, and labeling data of the tokens are configured to be stored in the above database. An electronic device that predicts risk ratings for mental disorders using spoken text.

9. In paragraph 1, The above first model, the above second model and the above third model are sequentially connected in series. An electronic device that predicts risk ratings for mental disorders using spoken text.

10. A method for predicting a risk grade for each mental illness using a speech text of an electronic device including a first model that inputs psychological counseling audio and outputs a transcribed speech text, a second model that inputs the speech text and outputs a symptom onset prediction score according to at least one of disease symptoms, risk factors, improvement factors, and intervention factors based on tokens and paragraphs in which the tokens are arranged, a third model that classifies the symptom onset prediction score for each disease and outputs a corresponding risk grade, and a memory and at least one processor, wherein the method comprises: A step in which the second model labels at least one of disease-specific symptoms, risk factors, improvement factors, and intervention factors for tokens in the utterance text as labeling data; A step of outputting a symptom onset prediction score for each disease by considering the weight of each labeling data according to the presence or absence or number of times of the labeling data; and A step in which the third model classifies the risk grade according to the symptom onset prediction score for each disease and outputs the classified risk grade for the entire disease as a single UI; A method for predicting risk ratings for mental disorders using spoken text.

Images