Method and system for psychological test based on brain signal analysis

Abstract

This system for a psychological test based on brain signal analysis comprises a brain signal measurement device, a signal processing module, a psychological test module, and a result output module. The brain signal measurement device generates an EEG signal by measuring an electrical signal generated from the brain of a user and generates fNIRS data by measuring a cerebral hemodynamic signal of the user. The signal processing module generates an fNIRS vector on the basis of the fNIRS data and generates an EEG vector on the basis of the EEG signal. The psychological test module uses fNIRS vectors and EEG vectors generated while the user inputs consent / non-consent responses to a plurality of psychological test problems, so as to estimate a response score indicating the degree of consent or non-consent of the user to each of the plurality of psychological test problems. The result output module analyzes the response score estimated for each of the plurality of psychological test problems, thereby providing psychological state determination results of the user.

Description

Psychological examination method and system based on brain signal analysis

[0001] The present invention relates to a psychological testing method, and more particularly, to a method and system for performing a psychological testing by analyzing signals generated in a user's brain while the user is responding to psychological testing questions.

[0002] Recently, many people are making great efforts to accurately understand their own psychological state in order to have smooth interpersonal relationships.

[0003] Typically, an accurate diagnosis of a person's current psychological state is made through consultation with a professional. However, such a diagnosis requires a visit to a hospital or counseling center. However, the counselor may feel psychologically burdened by visiting a hospital or counseling center. Furthermore, in-person consultations are limited in time and location, hindering the free flow of psychological assessment.

[0004] To this end, various types of psychological testing methods have been developed that allow users to conveniently assess their psychological state at any time and without being restricted by location through their terminals.

[0005] For example, self-report psychological tests such as the Myers-Briggs Type Indicator (MBTI), Minnesota Multiphasic Personality Inventory (MMPI), and BIG5 are widely used, in which users give themselves scores on given items ranging from 1 (strongly disagree) to k (strongly agree).

[0006] However, this self-report psychological test method using a multi-point scale response method has two problems.

[0007] The first problem is that users may sometimes respond in a way that contradicts their actual feelings. For example, users may give false answers to questions about things they feel negatively about, or they may respond in a way that reflects their desired self-image rather than their actual feelings.

[0008] A second problem is that the criteria for the rating scale are unclear. For example, it may be unclear how users distinguish between a 5 and a 4, and different people may have different criteria for selecting a 5 or a 4.

[0009] Due to the problems described above, self-report psychological tests using multi-point scale responses have the disadvantage of low test accuracy.

[0010] Accordingly, the technical problem of the present invention is conceived from this point, and one object of the present invention is to provide a brain signal analysis-based psychological test system and method that analyzes signals generated in the brain of a user while the user is responding to a psychological test question, estimates the user's degree of agreement or disagreement with the psychological test question, and uses the estimated degree of agreement or disagreement to estimate the user's response score for the psychological test question to determine the user's psychological state.

[0011] In order to achieve the above-described object of the present invention, a brain signal analysis-based psychological test system according to an embodiment of the present invention includes a brain signal measurement device, a signal processing module, a psychological test module, and a result output module. The brain signal measurement device is worn on a user's head, measures an electrical signal generated from the user's brain to generate an electroencephalogram (EEG) signal, and measures a hemodynamic signal generated by irradiating the user's brain with near-infrared light to calculate a concentration change of oxyhaemoglobin (HbO) and a concentration change of deoxyhaemoglobin (HbR) in the user's cerebral cortex, and generates functional Near Infra-Red Spectroscopy (fNIRS) data including the concentration change of oxyhaemoglobin and the concentration change of deoxyhaemoglobin. The signal processing module generates an fNIRS vector including a change in the concentration of the oxidized hemoglobin and a change in the concentration of the reduced hemoglobin for each time interval for each of a plurality of cerebral cortical regions based on the fNIRS data, and performs a short-time Fourier transform (STFT) on the EEG signal to generate an EEG vector including an amplitude value for each frequency for each of a plurality of time intervals. The psychological test module provides a plurality of psychological test questions requesting the user to select one of agree and disagree, and estimates a response score according to a multi-point scale indicating the degree of agreement or disagreement of the user for each of the plurality of psychological test questions by using the fNIRS vector and the EEG vector generated from the signal processing module while the user inputs an agree / disagree response for the plurality of psychological test questions.The above result output module analyzes the estimated response scores for each of the plurality of psychological test questions and provides a result of judging the psychological state of the user.

[0012] In order to achieve the above-described object of the present invention, a brain signal analysis-based psychological test method is performed by executing instructions stored in a computer-readable storage medium by a processor according to an embodiment of the present invention, wherein a plurality of psychological test questions are provided to a user who wears a brain signal measuring device that is worn on the head of a user and measures an electrical signal generated from the brain of the user to generate an electroencephalogram (EEG) signal, measures a hemodynamic signal generated by irradiating the brain of the user with near-infrared rays to calculate a concentration change of oxyhaemoglobin (HbO) and a concentration change of deoxyhaemoglobin (HbR) in the cerebral cortex of the user, and generates functional Near Infra-Red Spectroscopy (fNIRS) data including the concentration change of the oxyhaemoglobin and the concentration change of the deoxyhaemoglobin, and requests the user to select one of consent and non-consent, and the user wears the brain signal measuring device that is worn on the head of the user to measure an electrical signal generated from the brain of the user to generate an electroencephalogram (EEG) signal, and calculates a concentration change of oxyhaemoglobin (HbO) and a concentration change of deoxyhaemoglobin (HbR) in the cerebral cortex of the user and generates functional Near Infra-Red Spectroscopy (fNIRS) data including the concentration change of the oxyhaemoglobin and the concentration change of the deoxyhaemoglobin. While inputting the agree / disagree responses to a plurality of psychological test questions, an fNIRS vector including the change in the concentration of the oxidized hemoglobin and the change in the concentration of the reduced hemoglobin for each time interval for each of a plurality of cerebral cortical regions is generated based on the fNIRS data generated from the brain signal measurement device, and a short-time Fourier transform (SFT) is performed on the EEG signal.Performing STFT) to generate an EEG vector including amplitude values ​​for each frequency for each of a plurality of time intervals, estimating a response score according to a multi-point scale indicating the degree of agreement or disagreement with each of the plurality of psychological test questions of the user using the fNIRS vector and the EEG vector, and analyzing the estimated response score for each of the plurality of psychological test questions to provide a result of judging the psychological state of the user.

[0013] The brain signal analysis-based psychological test system and the brain signal analysis-based psychological test method performed by the brain signal analysis-based psychological test system according to embodiments of the present invention can estimate a multi-point scale response score (E_SC) indicating the degree to which the testee agrees or disagrees with the psychological test question by requesting the testee to only respond whether he or she agrees or disagrees with the psychological test question and analyzing the testee's brain signals generated while the testee responds whether he or she agrees or disagrees with the psychological test question.

[0014] Therefore, the accuracy of psychological tests can be effectively improved by eliminating the problems inherent in self-report psychological test methods using multi-point scale responses.

[0015] FIG. 1 is a diagram illustrating a brain signal analysis-based psychological examination system according to one embodiment of the present invention.

[0016] FIG. 2 is a block diagram showing an example of a psychological examination device included in the brain signal analysis-based psychological examination system of FIG. 1.

[0017] FIG. 3 is a flowchart illustrating a brain signal analysis-based psychological examination method according to one embodiment of the present invention.

[0018] FIG. 4 is a block diagram showing another example of a psychological examination device included in the brain signal analysis-based psychological examination system of FIG. 1.

[0019] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. Identical components in the drawings are designated by the same reference numerals, and redundant descriptions of identical components are omitted.

[0020] FIG. 1 is a diagram illustrating a brain signal analysis-based psychological examination system according to one embodiment of the present invention.

[0021] Referring to FIG. 1, a brain signal analysis-based psychological examination system (10) according to one embodiment of the present invention includes a psychological examination device (100) and a brain signal measurement device (200).

[0022] The brain signal measurement device (200) is worn on the head of a user undergoing a psychological test.

[0023] The brain signal measuring device (200) measures electrical signals generated from the user's brain to generate electroencephalogram (EEG) signals (EEG_S).

[0024] In addition, the brain signal measurement device (200) measures a hemodynamic signal generated by irradiating the user's brain with near-infrared rays to calculate a change in the concentration of oxyhaemoglobin (HbO) and a change in the concentration of deoxyhaemoglobin (HbR) in the user's cerebral cortex, and generates functional near-infra-red spectroscopy (fNIRS) data (fNIRS_D) including the change in the concentration of oxyhaemoglobin and the change in the concentration of deoxyhaemoglobin.

[0025] In one embodiment, the brain signal measuring device (200) may include a plurality of electrodes that measure electrical signals generated from the user's brain and a plurality of optodes that irradiate a laser of a near-infrared frequency band to the cerebral cortex and detect the laser that passes through the cerebral cortex and is emitted.

[0026] As illustrated in FIG. 1, the plurality of electrodes and the plurality of photoelectrodes can be distributed and arranged in a brain signal measurement device (200) to form a plurality of channels.

[0027] According to the anatomical structure of the human brain, several membranes exist beneath the skull, with thin gaps between each membrane. Therefore, when an electrical signal is generated in one area of ​​the brain, it propagates throughout the brain through the gaps between these membranes. This allows the electrical signal to be simultaneously measured by multiple electrodes forming multiple channels.

[0028] Therefore, the brain signal measurement device (200) can generate an EEG signal (EEG_S) using the average of the electrical signals measured in the plurality of channels through the plurality of electrodes.

[0029] Meanwhile, the photoelectrode includes a source that generates a laser in the near-infrared frequency band and a detector that detects the laser generated from the source, and a pair of the source and the detector can form a single channel. In this case, a pair of sources and detectors forming a single channel can be arranged at a distance of about 3 cm.

[0030] The brain signal measuring device (200) calculates the attenuation of the laser generated from the source, passes through the cerebral cortex, and is detected by the detector, and calculates the change in the concentration of oxyhaemoglobin (HbO) and the change in the concentration of deoxyhaemoglobin (HbR) in the cerebral cortex region located below the source and the detector forming one channel, and can generate fNIRS data (fNIRS_D) including the change in the concentration of oxyhaemoglobin and the change in the concentration of deoxyhaemoglobin.

[0031] Accordingly, the brain signal measurement device (200) can generate fNIRS data (fNIRS_D) for each of a plurality of cerebral cortex regions corresponding to the locations of the plurality of channels.

[0032] EEG signals (EEG_S) and fNIRS data (fNIRS_D) generated from a brain signal measurement device (200) can be provided to a psychological examination device (100).

[0033] Above, the configuration and operation of the brain signal measurement device (200) have been described, but the brain signal measurement device (200) included in the brain signal analysis-based psychological examination system (10) according to embodiments of the present invention can be implemented using various types of EEG-fNIRS simultaneous measurement devices known in the art that can simultaneously measure brain wave signals and cerebral hemodynamic signals to perform both EEG analysis and fNIRS analysis.

[0034] Since the configuration and operation of various types of existing EEG-fNIRS simultaneous measurement devices are widely known, a more detailed description of the detailed configuration and operation of the brain signal measurement device (200) is omitted here.

[0035] The psychological examination device (100) performs a self-report psychological examination on the user wearing the brain signal measurement device (200).

[0036] Self-report psychological tests are psychological tests that are conducted by having the test taker give a score on a multi-point scale for a given problem.

[0037] However, as described below, the psychological testing device (100) according to embodiments of the present invention does not request the user to give a score on a multi-point scale for a given question, as in a general self-reporting psychological testing method, but only requests the user to respond whether he or she agrees or disagrees with a given question.

[0038] For example, the psychological testing device (100) may ask the user to select only one of “yes” and “no” for a given question.

[0039] Specifically, the psychological test device (100) stores a problem set including a plurality of psychological test problems in a self-report type psychological test format, and can sequentially provide the plurality of psychological test problems to the user.

[0040] In one embodiment, the psychological testing device (100) can visually provide the psychological testing questions to the user through a display device.

[0041] In another embodiment, the psychological testing device (100) may provide the psychological testing questions to the user audibly through a speaker.

[0042] Thereafter, the psychological testing device (100) may ask the user to select whether to agree or disagree with the psychological testing questions provided to the user.

[0043] The psychological examination device (100) can receive the user's choice of whether to agree or disagree with the psychological examination question as an agree / disagree response.

[0044] According to an embodiment, the psychological examination device (100) can receive the consent / disagreement response from the user through various types of input devices such as a keyboard, mouse, microphone, etc.

[0045] Meanwhile, the psychological examination device (100) provides the user with the psychological examination question, and while the user inputs the agreement / disagreement response to the psychological examination question, the device can estimate a response score according to a multi-point scale indicating the degree of agreement or disagreement of the user with respect to the psychological examination question using the EEG signal (EEG_S) and fNIRS data (fNIRS_D) generated from the brain signal measurement device (200).

[0046] Thereafter, the psychological examination device (100) can analyze the response scores according to the estimated multi-point scale for each of the plurality of psychological examination questions and provide the result of judging the psychological state of the user.

[0047] In Fig. 1, the psychological examination device (100) is illustrated in the form of a server, but the present invention is not limited thereto, and the psychological examination device (100) can be implemented through various types of electronic devices capable of performing calculations.

[0048] Hereinafter, the configuration and operation of a brain signal analysis-based psychological examination system (10) according to embodiments of the present invention will be described in detail with reference to FIGS. 2 and 3.

[0049] FIG. 2 is a block diagram showing an example of a psychological examination device included in the brain signal analysis-based psychological examination system of FIG. 1.

[0050] FIG. 3 is a flowchart illustrating a brain signal analysis-based psychological examination method according to one embodiment of the present invention.

[0051] The brain signal analysis-based psychological examination method illustrated in FIG. 3 can be performed by the brain signal analysis-based psychological examination system (10) illustrated in FIGS. 1 and 2.

[0052] Hereinafter, with reference to FIGS. 2 and 3, the configuration and operation of the brain signal analysis-based psychological examination system (10) and the psychological examination method performed by the brain signal analysis-based psychological examination system (10) will be described in detail.

[0053] Referring to FIG. 2, the psychological test device (100) may include a signal processing module (SIGNAL PROCESSING MODULE) (110), a psychological test module (PSYCHOLOGICAL TEST MODULE) (120), and a result output module (RESULT OUTPUT MODULE) (130).

[0054] The term "module" as used herein may refer to software, hardware such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC), or a combination of software and hardware. A "module" may be stored in an addressable storage medium in the form of software and may be configured to be executed by one or more processors. For example, a "module" may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. A "module" may also be divided into multiple "modules" that perform specific functions.

[0055] The signal processing module (110) can receive EEG signals (EEG_S) and fNIRS data (fNIRS_D) from the brain signal measurement device (200).

[0056] As described above, the brain signal measurement device (200) can generate fNIRS data (fNIRS_D) including changes in the concentration of the oxidized hemoglobin and changes in the concentration of the reduced hemoglobin for each of the plurality of cerebral cortical regions corresponding to the positions of the plurality of channels formed by the plurality of photoelectrodes.

[0057] Accordingly, the signal processing module (110) can generate an fNIRS vector (fNIRS_V) including the change in the concentration of the oxidized hemoglobin and the change in the concentration of the reduced hemoglobin for each time interval for each of the plurality of cerebral cortical regions based on the fNIRS data (fNIRS_D).

[0058] For example, the signal processing module (110) may generate a first fNIRS vector (fNIRS_V) including the concentration change of the oxidized hemoglobin and the concentration change of the reduced hemoglobin measured for each time interval for the first cortical region, and may generate an r-th fNIRS vector (fNIRS_V) including the concentration change of the oxidized hemoglobin and the concentration change of the reduced hemoglobin measured for each time interval for the s-th cortical region. Here, r represents an integer greater than or equal to 2.

[0059] As described above, the brain signal measurement device (200) generates an EEG signal (EEG_S) using the average of the electrical signals measured in the plurality of channels through the plurality of electrodes, so the EEG signal (EEG_S) can correspond to an average brain wave signal for the entire brain of the user.

[0060] Accordingly, the signal processing module (110) can perform a short-time Fourier transform (STFT) on the EEG signal (EEG_S) to generate an EEG vector (EEG_V) including amplitude values ​​for each frequency for each of a plurality of time intervals.

[0061] For example, the signal processing module (110) may generate a first EEG vector (EEG_V) including frequency-specific amplitude values ​​calculated through a short-time Fourier transform for the EEG signal (EEG_S) in a first time interval, and may generate an s-th EEG vector (EEG_V) including frequency-specific amplitude values ​​calculated through a short-time Fourier transform for the EEG signal (EEG_S) in a s-th time interval. Here, s represents an integer greater than or equal to 2.

[0062] Brain waves generated in the human brain are classified into delta waves, theta waves, alpha waves, beta waves, and gamma waves according to the frequency band, and the EEG vector (EEG_V) generated from the signal processing module (110) may include amplitude values ​​for each frequency in the frequency band corresponding to delta waves, theta waves, alpha waves, beta waves, and gamma waves.

[0063] The fNIRS vector (fNIRS_V) and EEG vector (EEG_V) generated from the signal processing module (110) can be provided to the psychological test module (120).

[0064] The psychological test module (120) sequentially provides a plurality of psychological test questions to the user wearing the brain signal measurement device (200) requesting the user to select one of agree and disagree, and estimates a response score (E_SC) according to a multi-point scale indicating the degree of agreement or disagreement of the user for each of the plurality of psychological test questions using the fNIRS vector (fNIRS_V) and EEG vector (EEG_V) generated from the signal processing module (110) while the user inputs an agree / disagree response (YN) indicating whether he or she agrees or disagrees with the plurality of psychological test questions.

[0065] Below, the operation of the psychological examination module (120) is described in detail.

[0066] It is widely known that people generally produce different brain signals according to fNIRS depending on whether they agree or disagree with the perceived information.

[0067] For example, according to "Interpersonal Agreement and Disagreement During Face-to-Face Dialogue: An fNIRS Investigation", Hum. Neurosci., 13 January 2021, (Hirsch, et al.), when analyzing the brain signals of people having a face-to-face conversation using fNIRS, it was confirmed that the brain signals generated according to fNIRS analysis were different when agreeing with what the other person said and when disagreeing.

[0068] Accordingly, the psychological test module (120) can generate an agreement judgment model to estimate whether the user agrees or disagrees with the psychological test questions provided to the user using the fNIRS vector (fNIRS_V).

[0069] In one embodiment, the psychological test module (120) can perform deep learning on an artificial neural network to create the consent judgment model.

[0070] Specifically, the psychological test module (120) provides a proposition with a clear true or false answer as a learning problem (QT) to each of a plurality of learners wearing a brain signal measurement device (200), and then generates input data using fNIRS vectors (fNIRS_V) generated from the signal processing module (110) for a first time, and can generate first learning data that labels the input data with whether the learning problem (QT) is actually true or false.

[0071] For example, the psychological test module (120) can use propositions with clear true or falsehood, such as “I am a man” and “The Earth is square,” as learning problems (QT).

[0072] As described above, since the learning problem (QT) is a proposition that is clearly true or false, the learner can recognize whether the learning problem (QT) is true or false as soon as he or she confirms the learning problem (QT), and the hemodynamic signals generated in the learner's brain may be different depending on whether he or she recognized it as true or false.

[0073] Therefore, considering the time it takes for the learner to read the given learning problem (QT), the first time can be set to 2 seconds or less.

[0074] However, the present invention is not limited thereto, and the first time can be set flexibly depending on the length of the learning problem (QT).

[0075] Meanwhile, as described above, the signal processing module (110) can generate an fNIRS vector (fNIRS_V) for each of the plurality of cerebral cortical regions.

[0076] In one embodiment, the psychological test module (120) may determine a matrix generated by concatenating the fNIRS vectors (fNIRS_V) for each of the plurality of cerebral cortical regions generated from the signal processing module (110) during the first time period as the input data.

[0077] In another embodiment, the psychological test module (120) may determine, as the input data, a vector corresponding to the average of the fNIRS vectors (fNIRS_V) for each of the plurality of cerebral cortical regions generated from the signal processing module (110) during the first time.

[0078] The psychological test module (120) can generate the first learning data, and then, when inputting an fNIRS vector into an artificial neural network using the first learning data, the artificial neural network can perform binary classification to classify whether the input fNIRS vector is true or false, and train the artificial neural network to output an agreement / disagreement estimation response indicating whether the input fNIRS vector is true or false, thereby generating the agreement / disagreement judgment model.

[0079] The artificial neural network used to create the above consent judgment model may be any artificial neural network capable of performing binary classification.

[0080] Meanwhile, the psychological examination module (120) may exclude the input data from the first learning data if there is a possibility that the input data contains incorrect information in order to improve the accuracy of the consent judgment model.

[0081] For example, the psychological test module (120) may provide the learner with a learning question (QT) and, after the first time has elapsed, ask the learner to select whether he or she agrees or disagrees with the learning question (QT).

[0082] If the agree / disagree response (YN) entered by the learner for the learning question (QT) is different from the actual truth or falsehood of the learning question (QT), the learner has responded in the opposite direction to a proposition whose true or falseness is clear, and therefore the input data corresponding to the learner's learning question (QT) may contain incorrect information.

[0083] In this case, the psychological test module (120) can exclude the input data corresponding to the learning problem (QT) from the first learning data.

[0084] In this way, if the agree / disagree response (YN) input by the learner for the learning problem (QT) is different from the actual true or false of the learning problem (QT), the psychological test module (120) excludes the input data corresponding to the learner's learning problem (QT) from the first learning data, and then trains an artificial neural network using the first learning data to generate the agree / disagree judgment model, thereby effectively improving the accuracy of the agree / disagree judgment model.

[0085] It is widely known that people's brains produce different EEG signals depending on how confident they are about their own judgments.

[0086] For example, according to "Measuring Human Decision Confidence from EEG Signals in an Object Detection Task", 10th International IEEE / EMBS Conference on Neural Engineering (NER) (2021) (Li, et al.), when people were given images and asked to find specific objects in the images, and the EEG signals generated in the people's brains were analyzed, it was confirmed that the EEG signals were generated differently depending on how confident people were about the objects they had found.

[0087] Accordingly, the psychological test module (120) can generate a confidence level judgment model to estimate the degree to which the user agrees or disagrees with the psychological test questions provided to the user using the EEG vector (EEG_V).

[0088] In one embodiment, the psychological test module (120) can perform deep learning on an artificial neural network to generate the confidence level judgment model.

[0089] Specifically, the psychological test module (120) provides a problem related to the psychological test as a learning problem (QT) to each of a plurality of learners wearing a brain signal measuring device (200), and in order to give the learner time to read and understand the learning problem (QT), after a first time has passed from the time the learning problem (QT) was provided to the learner, the learner may be requested to input an agree / disagree response (YN) indicating whether he or she agrees or disagrees with the learning problem (QT).

[0090] For example, the psychological test module (120) can use sentences with various contents related to psychological tests as learning problems (QT), such as “I sometimes feel great sadness,” “I get along well with my friends,” and “I like being alone at home.”

[0091] The psychological test module (120) may generate input data using EEG vectors (EEG_V) generated from the signal processing module (110) for a second time after requesting the learner to input an agree / disagree response (YN) to the learning problem (QT).

[0092] Typically, when a learner is asked to enter an Agree / Disagree response (YN) to a learning question (QT), he or she will reflect on whether he or she agrees or disagrees with the learning question (QT) and then enter an Agree / Disagree response (YN).

[0093] Additionally, while brain waves generally respond quickly to stimulation, cerebral blood flow responses, in which the concentrations of oxygenated and reduced hemoglobin change in response to stimulation, can occur after a time delay of about 3 seconds.

[0094] Therefore, taking into account the above time delay and the time for the learner to consider whether he or she agrees or disagrees with the learning question (QT), the second time can be set to about 5 seconds.

[0095] However, the present invention is not limited thereto, and the second time may be set flexibly depending on the difficulty of the learning problem (QT).

[0096] Meanwhile, as described above, the signal processing module (110) can generate an EEG vector (EEG_V) for each of the plurality of time intervals.

[0097] In one embodiment, the psychological examination module (120) may determine a matrix generated by concatenating EEG vectors (EEG_V) for each of the plurality of time intervals generated from the signal processing module (110) during the second time period as the input data.

[0098] In another embodiment, the psychological examination module (120) may determine, as the input data, a vector corresponding to the average of the EEG vectors (EEG_V) for each of the plurality of time intervals generated from the signal processing module (110) during the second time period.

[0099] After the learner inputs an agree / disagree response (YN), the psychological test module (120) may request the learner to select a confidence level (SC) representing the degree of confidence in the agree / disagree response from among the first to nth levels, and may generate second learning data using the confidence level (SC) selected by the learner as a label for the input data. Here, n represents an integer greater than or equal to 2.

[0100] The psychological test module (120) can generate the second learning data, and then, when inputting an EEG vector into an artificial neural network using the second learning data, the artificial neural network can perform multi-label classification to classify the input EEG vector into one of the first to n-th levels, thereby training the artificial neural network to output an estimated confidence level corresponding to the input EEG vector among the first to n-th levels, thereby generating the confidence level judgment model.

[0101] The artificial neural network used to generate the above confidence level judgment model may be any artificial neural network capable of performing multi-label classification.

[0102] The psychological test module (120) can perform a psychological test on the user by generating the consent judgment model and the confidence level judgment model through the operations described above, and then using a problem set including a plurality of psychological test problems and the consent judgment model and the confidence level judgment model.

[0103] According to an embodiment, the problem set including the plurality of psychological test problems may be stored in advance within the psychological test device (100) or may be provided from the outside.

[0104] As described above with reference to FIG. 1, the above-described multiple psychological test questions are questions for a self-report psychological test, but unlike a typical self-report psychological test method, the user is not asked to give a score on a multi-point scale for the given question, but is asked only to respond whether he or she agrees or disagrees with the given question.

[0105] The psychological test module (120) can sequentially provide the user wearing the brain signal measurement device (200) with a plurality of psychological test questions requesting the user to select one of consent and non-consent (step S100).

[0106] The signal processing module (110) can generate an fNIRS vector (fNIRS_V) and an EEG vector (EEG_V) by performing the operations described above while the user inputs an agree / disagree response (YN) indicating whether he or she agrees or disagrees with the plurality of psychological test questions (step S200).

[0107] The psychological test module (120) can estimate a response score (E_SC) according to a multi-point scale indicating the degree of agreement or disagreement of the user for each of the plurality of psychological test questions by using the fNIRS vector (fNIRS_V) and the EEG vector (EEG_V) generated from the signal processing module (110) while the user inputs an agree / disagree response (YN) indicating whether he or she agrees or disagrees with the plurality of psychological test questions, and the agreement / disagreement judgment model and the confidence level judgment model (step S300).

[0108] Specifically, the psychological test module (120) can provide the psychological test question to the user wearing the brain signal measurement device (200), and generate consent / disagreement input data using the fNIRS vectors (fNIRS_V) generated from the signal processing module (110) during the first time after providing the psychological test question.

[0109] The psychological test module (120) can generate the consent / disagreement input data in the same manner as the process of generating the input data by using the fNIRS vectors (fNIRS_V) generated from the signal processing module (110) during the first time after providing the learner with a learning problem (QT) when generating the first learning data to generate the consent / disagreement judgment model.

[0110] In one embodiment, the psychological test module (120) may determine a matrix generated by concatenating the fNIRS vectors (fNIRS_V) for each of the plurality of cerebral cortical regions generated from the signal processing module (110) during the first time as the consent / disagreement input data.

[0111] In another embodiment, the psychological test module (120) may determine a vector corresponding to the average of the fNIRS vectors (fNIRS_V) for each of the plurality of cerebral cortical regions generated from the signal processing module (110) during the first time as the consent / disagreement input data.

[0112] Thereafter, the psychological test module (120) can input the above-mentioned consent / disagreement input data into the above-mentioned consent / disagreement judgment model and obtain the above-mentioned consent / disagreement estimated response output from the above-mentioned consent / disagreement judgment model.

[0113] Therefore, the above agreed / disagreed response can indicate whether the user agrees or disagrees with the psychological test question.

[0114] In addition, the psychological test module (120) may request the user to select whether to agree or disagree with the psychological test question after the first time period has elapsed from the time the psychological test question is provided to the user, and may receive the user's agreement / disagree response to the psychological test question, and may generate confidence level input data using EEG vectors (EEG_V) generated from the signal processing module (110) during the second time period after requesting the user to select whether to agree or disagree with the psychological test question.

[0115] When generating the second learning data to generate the confidence level judgment model, the psychological test module (120) may generate the confidence level input data in the same manner as the process of generating the input data by using EEG vectors (EEG_V) generated from the signal processing module (110) during the second time after requesting the learner to select whether to agree or disagree with the learning problem (QT).

[0116] In one embodiment, the psychological test module (120) may determine a matrix generated by concatenating EEG vectors (EEG_V) for each of the plurality of time intervals generated from the signal processing module (110) during the second time period as the confidence level input data.

[0117] In another embodiment, the psychological test module (120) may determine, as the confidence level input data, a vector corresponding to the average of the EEG vectors (EEG_V) for each of the plurality of time intervals generated from the signal processing module (110) during the second time period.

[0118] Thereafter, the psychological test module (120) can input the confidence level input data into the confidence level judgment model and obtain the estimated confidence level output from the confidence level judgment model.

[0119] Therefore, the estimated confidence level is a value corresponding to one of the first to nth levels, and represents the degree to which the user agrees or disagrees with the psychological test question.

[0120] The psychological test module (120) obtains the estimated agreement / disagreement response output from the agreement / disagreement judgment model and the estimated confidence level output from the confidence level judgment model, and then estimates a response score (E_SC) according to a (2*n) point scale indicating the degree of agreement or disagreement of the user with respect to the psychological test question based on the estimated agreement / disagreement response and the estimated confidence level.

[0121] Specifically, the psychological test module (120) can determine the response score (E_SC) for the psychological test question as (n-p+1) points when the estimated agreement / disagreement response for the psychological test question corresponds to disagreement and the estimated confidence level for the psychological test question corresponds to the p level. Here, p represents a positive integer less than or equal to n.

[0122] In addition, the psychological test module (120) can determine the response score (E_SC) for the psychological test question as (n+q) points when the estimated agreement / disagreement response for the psychological test question corresponds to agreement and the estimated confidence level for the psychological test question corresponds to level q. Here, q represents a positive integer less than or equal to n.

[0123] Accordingly, when the response score (E_SC) for the psychological test question generated from the psychological test module (120) is 1 to n points, it can be estimated that the user disagrees with the psychological test question, and the degree of certainty about the disagreement increases as the response score (E_SC) decreases.

[0124] In addition, when the response score (E_SC) for the psychological test question generated from the psychological test module (120) is (n+1) points to (2*n) points, it can be estimated that the user agrees with the psychological test question, and the degree of confidence in the agreement increases as the response score (E_SC) increases.

[0125] That is, if the response score (E_SC) for the above psychological test question is 1 point, the user is estimated to very strongly disagree with the above psychological test question, and if the response score (E_SC) for the above psychological test question is (2*n) points, the user is estimated to very strongly agree with the above psychological test question.

[0126] The psychological test module (120) can provide an estimated response score (E_SC) for each of the plurality of psychological test questions to the result output module (130).

[0127] The result output module (130) can analyze the estimated response score (E_SC) for each of the plurality of psychological test questions provided to the user and provide the result of judging the psychological state of the user (step S400).

[0128] In one embodiment, the result output module (130) may pre-store a psychological analysis algorithm that outputs a corresponding psychological state when a (2*n)-point scale score is given for each of the plurality of psychological test questions.

[0129] The above psychological analysis algorithm may be one of various types of psychological analysis algorithms using a multi-point scale response method known in the past.

[0130] Accordingly, the result output module (130) can input the response score (E_SC) for each of the plurality of psychological test questions provided from the psychological test module (120) into the psychological analysis algorithm to provide the result of the psychological state judgment of the user.

[0131] Meanwhile, in order to improve the accuracy of estimating the degree of agreement or disagreement of the user with respect to the psychological test question, the psychological test module (120) may remove the estimated response score (E_SC) for the psychological test question if it is determined that the user has intentionally given a false response to the psychological test question, such as when the user responds differently from his / her actual psychology to the psychological test question, when the user responds in the opposite way to a question related to something he / she feels negative about, or when the user responds in accordance with his / her desired appearance rather than his / her actual psychology.

[0132] In one embodiment, if the agree / disagree response (YN) entered by the user for the psychological test question does not match the estimated agree / disagree response for the psychological test question, the psychological test module (120) may provide the psychological test question to the user again and re-request the user to select one of agree and disagree for the psychological test question.

[0133] In another embodiment, if the user's input of an agree / disagree response (YN) for the psychological test question matches the estimated agree / disagree response for the psychological test question, the psychological test module (120) provides the estimated response score (E_SC) for the psychological test question to the result output module (130), and if the user's input of an agree / disagree response (YN) for the psychological test question does not match the estimated agree / disagree response for the psychological test question, the psychological test module (120) may remove the estimated response score (E_SC) for the psychological test question without providing it to the result output module (130).

[0134] In this case, the result output module (130) can provide the result of the psychological state judgment of the user using only the response score (E_SC) provided from the psychological test module (120).

[0135] Meanwhile, if there are a large number of psychological test questions among the plurality of psychological test questions for which the agree / disagree response (YN) entered by the user and the estimated agree / disagree response to the psychological test question do not match, it can be assumed that the user intentionally gave false responses to a large number of psychological test questions.

[0136] Accordingly, if the number of psychological test questions among the plurality of psychological test questions for which the user's inputted agree / disagree response (YN) and the estimated agree / disagree response to the psychological test question do not match is greater than a threshold number, the psychological test module (120) may end the psychological test for the user without providing all estimated response scores (E_SC) for the plurality of psychological test questions to the result output module (130).

[0137] FIG. 4 is a block diagram showing another example of a psychological examination device included in the brain signal analysis-based psychological examination system of FIG. 1.

[0138] FIG. 4 illustrates a case where the signal processing module (110), psychological examination module (120), and result output module (130) of FIG. 2 are all implemented in software form.

[0139] Referring to FIG. 4, the psychological examination device (1000) includes a processor (1100), an input / output (I / O) device (1200), a network interface (NETWORK INTERFACE) (1300), a random access memory (RAM) (1400), a read only memory (ROM) (1500), and a storage device (STORAGE) (1600).

[0140] The processor (1100) can access memory, i.e., RAM (1400) or ROM (1500), through a bus, and execute instructions stored in the RAM (1400) or ROM (1500). As illustrated in FIG. 4, the RAM (1400) can store all or part of a program (PR) corresponding to the signal processing module (110), the psychological test module (120), and the result output module (130) of FIG. 2, and the program (PR) can cause the processor (1100) to perform psychological test operations (e.g., steps S100, S200, S300, and S400 of FIG. 3).

[0141] The storage device (1600) can store a program (PR), and all or part of the program (PR) can be loaded from the storage device (1600) into the RAM (1400) before the program (PR) is executed by the processor (1100). The storage device (1600) can also store a file written in a programming language, and all or part of a program (PR) generated by a compiler or the like can be loaded into the RAM (1400).

[0142] Additionally, the storage device (1600) may store various data required for performing the artificial neural network and deep learning described above with reference to FIGS. 1 to 3.

[0143] The input / output device (1200) may include input devices such as a keyboard, microphone, pointing device, etc., and may include output devices such as a display device, speaker, printer, etc. For example, an administrator may trigger the execution of a program (PR) by the processor (1100) through the input / output device (1200), and may also provide or confirm various inputs / data.

[0144] The network interface (1300) may provide access to a network external to the psychological testing device (1000). For example, the network may include multiple computing systems and communication links, which may include wired, optical, wireless, or any other form of link. Various inputs may be provided to the psychological testing device (1000) through the network interface (1300), and various outputs may be provided to other computing systems through the network interface (1300).

[0145] Additionally, the network interface (1300) may also receive EEG signals (EEG_S) and fNIRS data (fNIRS_D) from the brain signal measurement device (200).

[0146] In one embodiment, the computer program instructions, signal processing module (110), psychological testing module (120), and result output module (130) may be stored on a computer-readable medium. Additionally, various intermediate data and result data generated during the computational processing performed by the processor (1100) may be stored on a computer-readable medium.

[0147] In one embodiment, the psychological testing device (1000) of FIG. 4 may be implemented in the form of various electronic systems such as a personal computer (PC), a server computer, a data center, a workstation, a laptop, a cellular phone, a smart phone, etc.

[0148] In general, self-report psychological testing methods using multi-point scale responses have the problem that the test taker may submit false responses that are not based on his or her actual psychology, and that the reference point of the score scale is unclear.

[0149] However, as described above with reference to FIGS. 1 to 4, the brain signal analysis-based psychological test system (10) according to embodiments of the present invention, although the plurality of psychological test questions provided to the user are questions for a self-report psychological test, does not request the user to give a score on a (2*n)-point scale for the psychological test questions as in a general self-report psychological test method, but requests only a response as to whether or not the user agrees or disagrees with the psychological test questions, and analyzes the user's brain signals generated while the user responds as to whether or not he or she agrees or disagrees with the psychological test questions, thereby estimating a response score (E_SC) on a (2*n)-point scale indicating the degree to which the user agrees or disagrees with the psychological test questions.

[0150] Therefore, the brain signal analysis-based psychological test system (10) and the brain signal analysis-based psychological test method performed by the brain signal analysis-based psychological test system (10) according to embodiments of the present invention can effectively improve the accuracy of the psychological test by eliminating the above-mentioned problems of the self-report psychological test method of the multi-point scale response method.

[0151] The present invention can be usefully utilized to improve the accuracy of self-report psychological tests using a multi-point scale response method.

[0152] As described above, although the present invention has been described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various modifications and changes may be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below.

Claims

1. A brain signal measuring device that is worn on a user's head and measures electrical signals generated from the user's brain to generate electroencephalogram (EEG) signals, measures hemodynamic signals generated by irradiating the user's brain with near-infrared rays to calculate changes in the concentration of oxyhaemoglobin (HbO) and deoxyhaemoglobin (HbR) in the user's cerebral cortex, and generates functional Near Infra-Red Spectroscopy (fNIRS) data including changes in the concentration of oxyhaemoglobin and deoxyhaemoglobin; A signal processing module that generates an fNIRS vector including a change in the concentration of the oxidized hemoglobin and a change in the concentration of the reduced hemoglobin for each time interval for each of a plurality of cerebral cortical regions based on the fNIRS data, and performs a short-time Fourier transform (STFT) on the EEG signal to generate an EEG vector including an amplitude value for each frequency for each of a plurality of time intervals; A psychological test module that provides a plurality of psychological test questions to the user, requesting the user to select one of agree and disagree, and estimates a response score according to a multi-point scale indicating the degree of agreement or disagreement of the user for each of the plurality of psychological test questions using the fNIRS vector and the EEG vector generated from the signal processing module while the user inputs an agree / disagree response to the plurality of psychological test questions; and A brain signal analysis-based psychological test system including a result output module that analyzes the estimated response scores for each of the plurality of psychological test questions and provides a result of judging the psychological state of the user.

2. In the first paragraph, the psychological examination module, After providing a proposition with a clear true or false answer as a learning problem to each of a plurality of learners wearing the brain signal measurement device, input data is generated using the fNIRS vectors generated from the signal processing module for a first time, and first learning data is generated with a label for the input data indicating whether the learning problem is actually true or false. A brain signal analysis-based psychological examination system that generates a consent judgment model by training the artificial neural network to perform binary classification to classify whether the input fNIRS vector is true or false when inputting an fNIRS vector into the artificial neural network using the first learning data, thereby outputting an agreement / disagreement estimation response indicating whether the input fNIRS vector is true or false.

3. In the second paragraph, the psychological examination module, A brain signal analysis-based psychological examination system that determines a matrix generated by connecting the fNIRS vectors for each of the plurality of cerebral cortex regions generated from the signal processing module during the first time as the input data.

4. In the second paragraph, the psychological examination module, A brain signal analysis-based psychological examination system that determines, as input data, a vector corresponding to the average of the fNIRS vectors for each of the plurality of cerebral cortical regions generated from the signal processing module during the first time period.

5. In the second paragraph, the psychological examination module, A brain signal analysis-based psychological test system that provides the learning problem to the learner, asks the learner to select whether to agree or disagree with the learning problem after the first period of time has elapsed, and excludes the input data corresponding to the learning problem from the first learning data if the agree / disagree response entered by the learner for the learning problem is different from whether the learning problem is actually true or false.

6. In the second paragraph, the psychological examination module, Providing a problem related to a psychological test as a learning problem to each of a plurality of learners wearing the brain signal measuring device, and after a first period of time, requesting the learner to input an agree / disagree response indicating whether he or she agrees or disagrees with the learning problem, and after requesting the learner to input the agree / disagree response, generating input data using the EEG vectors generated from the signal processing module for a second period of time; After the learner inputs an agree / disagree response (YN), the learner is asked to select a confidence level representing the degree of confidence in the agree / disagree response from among levels 1 to n (where n is an integer greater than or equal to 2), and second learning data is generated using the confidence level selected by the learner as a label for the input data. A brain signal analysis-based psychological examination system that generates a confidence level judgment model by training the artificial neural network to perform multi-label classification to classify the input EEG vector into one of the first to n-th levels when inputting an EEG vector to the artificial neural network using the second learning data, and outputting an estimated confidence level corresponding to the input EEG vector among the first to n-th levels.

7. In paragraph 6, the psychological examination module, A brain signal analysis-based psychological examination system that determines a matrix generated by connecting the EEG vectors for each of the plurality of time intervals generated from the signal processing module during the second time as the input data.

8. In paragraph 6, the psychological examination module, A brain signal analysis-based psychological examination system that determines, as input data, a vector corresponding to the average of the EEG vectors for each of the plurality of time intervals generated from the signal processing module during the second time period.

9. In paragraph 6, the psychological examination module, Sequentially providing each of the plurality of psychological test questions to the user wearing the brain signal measurement device, generating consent / disagreement input data using the fNIRS vectors generated from the signal processing module for a first time after providing the psychological test questions, and inputting the data into the consent / disagreement judgment model, and obtaining the consent / disagreement estimation response output from the consent / disagreement judgment model. After the first time has elapsed, requesting the user to select whether to agree or disagree with the psychological test question, receiving the user's agreement / disagree response to the psychological test question, and after requesting the user to select whether to agree or disagree with the psychological test question, generating confidence level input data using the EEG vectors generated from the signal processing module for a second time and inputting the confidence level judgment model, and obtaining the estimated confidence level output from the confidence level judgment model. If the estimated agreement / disagreement response to the above psychological test question corresponds to disagreement, and the estimated confidence level for the above psychological test question corresponds to the p level (where p is a positive integer less than or equal to n), the response score for the above psychological test question is determined as (n-p+1) points, A brain signal analysis-based psychological test system that determines the response score for the psychological test question as (n+q) points when the estimated agreement / disagreement response for the psychological test question corresponds to agreement and the estimated confidence level for the psychological test question corresponds to level q (q is a positive integer less than or equal to n).

10. In paragraph 9, the psychological examination module, A brain signal analysis-based psychological test system that, if the agreement / disagreement response entered by the user for the above psychological test question does not match the estimated agreement / disagreement response for the above psychological test question, provides the psychological test question to the user again and re-requests the user to select one of agreement and disagreement for the psychological test question.

11. In paragraph 9, the psychological examination module, If the user's agreed / disagreed response to the psychological test question matches the estimated agreed / disagreed response to the psychological test question, the estimated response score for the psychological test question is provided to the result output module. A brain signal analysis-based psychological test system that does not provide the estimated response score for the psychological test question to the result output module when the agreement / disagreement response entered by the user for the psychological test question does not match the estimated agreement / disagreement response for the psychological test question.

12. In paragraph 9, the psychological examination module, A brain signal analysis-based psychological test system that terminates the psychological test for the user without providing the estimated response scores for the plurality of psychological test questions to the result output module, when the number of psychological test questions for which the user's agreed / disagreed response and the estimated agreed / disagreed response to the psychological test question do not match is greater than a threshold number among the plurality of psychological test questions.

13. A brain signal analysis-based psychological test method performed by executing instructions stored in a computer-readable storage medium by a processor, A step of providing a plurality of psychological test questions to a user wearing a brain signal measuring device that is worn on the head of the user and measures electrical signals generated from the brain of the user to generate an electroencephalogram (EEG) signal, measures hemodynamic signals generated by irradiating the brain of the user with near-infrared rays to calculate changes in the concentration of oxyhaemoglobin (HbO) and changes in the concentration of deoxyhaemoglobin (HbR) in the cerebral cortex of the user, and generates functional Near Infra-Red Spectroscopy (fNIRS) data including changes in the concentration of oxyhaemoglobin and changes in the concentration of deoxyhaemoglobin, and requests the user to select one of consent and non-consent; A step of generating an fNIRS vector including a change in the concentration of the oxidized hemoglobin and a change in the concentration of the reduced hemoglobin for each time interval for each of a plurality of cerebral cortical regions based on the fNIRS data generated from the brain signal measurement device while the user inputs an agreement / disagreement response to the plurality of psychological test questions, and performing a short-time Fourier transform (STFT) on the EEG signal to generate an EEG vector including an amplitude value for each frequency for each of the plurality of time intervals; A step of estimating a response score according to a multi-point scale indicating the degree of agreement or disagreement with each of the plurality of psychological test questions of the user using the fNIRS vector and the EEG vector; and A brain signal analysis-based psychological test method comprising a step of analyzing the estimated response scores for each of the plurality of psychological test questions and providing a result of judging the psychological state of the user.

Images