Information processing device and computer program
The information processing device and computer program use machine learning to analyze brain activity patterns, addressing the challenge of predicting compatibility by calculating correlation matrices and feature vectors, thereby enhancing the accuracy of compatibility predictions.
Patent Information
- Application Number
- JP2020188808
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-11-12
- Publication Date
- 2024-11-20
- Estimated Expiration
- 2040-11-12
AI Technical Summary
Existing technologies lack the ability to accurately predict compatibility between individuals based on their brain activity patterns.
An information processing device and computer program that utilize machine learning to analyze time-series data from multiple brain regions, calculating correlation matrices and feature vectors to predict compatibility by evaluating functional connectivity differences between individuals.
Enables accurate prediction of compatibility between individuals, leveraging machine learning to identify patterns in brain activity that correlate with social interactions.
Smart Images

Figure 0007587232000004 
Figure 0007587232000005 
Figure 0007587232000006
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an information processing device and a computer program. [Background technology]
[0002] Conventionally, research on human brain functions has been conducted using electroencephalograms, brain magnetic fields, magnetic resonance images, etc. For example, Non-Patent Document 1 mentions the possibility that there are individual differences in the frequency of brain activity, and that there is a kind of compatibility in which brain activity is easier or harder to synchronize depending on the combination of individuals. Non-Patent Document 1 also mentions the existence of knowledge that the closer the friendship between people, the higher the similarity of their resting brain activity patterns when examined using fMRI data. [Prior art documents] [Patent documents]
[0003] [Non-Patent Document 1] Psychological Review 2016, Vol. 59, No. 3, pp. 283-291, Hideki Ohira, "Mechanisms that lead to motivation in brain activity" Summary of the Invention [Problem to be solved by the invention]
[0004] If it were possible to predict the compatibility between two people before they meet, this could be used in a variety of situations. The present disclosure provides techniques for accurately predicting compatibility between two people. [Means for solving the problem]
[0005] An information processing device according to an embodiment of the present disclosure is an information processing device for predicting compatibility between humans. The information processing device includes an arithmetic circuit, a storage device that stores a prediction model, an input interface that acquires time-series data indicating changes in human brain activity, and an output interface that outputs a prediction result of compatibility between humans. The human brain includes a plurality of brain regions, and each of the plurality of brain regions is associated with each of a plurality of functions possessed by the brain. The time-series data includes data indicating temporal changes in activity of each of the plurality of brain regions acquired from each of the plurality of brain regions. The input interface acquires first time-series data and second time-series data, which are time-series data of a first human and a second human. The arithmetic circuit uses each of the first time-series data and the second time-series data to calculate a correlation coefficient indicating the degree of functional coupling between the brain regions for each combination of two brain regions selected from the plurality of brain regions, thereby generating a first correlation matrix, which is a set of correlation coefficients between the brain regions of the first human, and a second correlation matrix, which is a set of correlation coefficients between the brain regions of the second human. The arithmetic circuit also uses the first correlation matrix and the second correlation matrix to calculate a feature vector indicating a difference in functional connections between the brain regions of the first person and the second person, and inputs the feature vector into a prediction model to predict the compatibility between the first person and the second person. The prediction model is constructed by performing machine learning using at least a first type of teacher data that is a combination of a feature vector indicating a difference in functional connections between the brain regions of two people who are known in advance to have good compatibility and a first compatibility label indicating good compatibility, and a second type of teacher data that is a combination of a feature vector indicating a difference in functional connections between the brain regions of two people who are known in advance to not have good compatibility and a second compatibility label indicating not good compatibility.
[0006] A computer program according to an embodiment of the present disclosure is a computer program executed by an arithmetic circuit of an information processing device to cause the information processing device to predict compatibility between people. The information processing device includes an arithmetic circuit, a storage device that stores a prediction model, an input interface that acquires time-series data indicating changes in human brain activity, and an output interface that outputs a prediction result of compatibility between people. The human brain includes a plurality of brain regions, each of which is associated with a plurality of functions possessed by the brain. The time-series data includes data acquired from each of the plurality of brain regions that indicates changes in activity of each brain region over time. The computer program causes the arithmetic circuit to acquire first time-series data and second time-series data, which are time-series data of a first person and a second person, via the input interface. The computer program also causes the arithmetic circuit to use each of the first time-series data and the second time-series data to calculate a correlation coefficient indicating the degree of functional coupling between the brain regions for each combination of two brain regions selected from the plurality of brain regions, thereby generating a first correlation matrix which is a set of correlation coefficients between the brain regions of the first person and a second correlation matrix which is a set of correlation coefficients between the brain regions of the second person. Furthermore, the computer program causes the arithmetic circuit to use the first correlation matrix and the second correlation matrix to calculate a feature vector indicating a difference in functional coupling between the brain regions between the first person and the second person, and inputs the feature vector into a prediction model to predict the compatibility between the first person and the second person. The predictive model is constructed by machine learning using at least a first type of training data which is a combination of a feature vector indicating differences in functional connectivity between brain regions between two people who are known in advance to have good compatibility and a first compatibility label indicating good compatibility, and a second type of training data which is a combination of a feature vector indicating differences in functional connectivity between brain regions between two people who are known in advance to not have good compatibility and a second compatibility label indicating not good compatibility.
[0007] A trained prediction model according to an embodiment of the present disclosure is a trained prediction model for predicting compatibility between a first person and a second person. The prediction model is constructed by performing machine learning using at least a first type of teacher data, which is a combination of a feature vector indicating a difference in functional connections between brain regions between two people who are known to have good compatibility in advance and a first compatibility label indicating good compatibility, and a second type of teacher data, which is a combination of a feature vector indicating a difference in functional connections between brain regions between two people who are known to have poor compatibility in advance and a second compatibility label indicating poor compatibility. When a feature vector indicating a difference in functional connections between brain regions between the first person and the second person is input, the prediction model outputs the compatibility between the first person and the second person as a prediction result. Effect of the Invention
[0008] According to the present disclosure, it becomes possible to accurately predict the compatibility between two people. [Brief description of the drawings]
[0009] [Figure 1] A schematic diagram showing the speed dating venue [Figure 2A] A diagram showing an example of the results of tallying answers to questions [Figure 2B] A diagram showing examples of the relationship between each pair of men and women and their compatibility [Diagram 3] An example of 100 fMRI images of a participant's brain. [Figure 4] A diagram showing four frequency bands and the waveforms that belong to each frequency band. [Diagram 5] FIG. 1 shows an example of a procedure for calculating a difference feature vector for evaluating functional connectivity [Figure 6] Diagram showing the process flow during machine learning training [Figure 7A] A graph showing the accuracy rate of machine learning-based classification for each frequency [Figure 7B]A diagram showing the results of an analysis of the validity of compatibility prediction results for frequency bands that contain compatibility information. [Figure 8] Schematic diagram of a compatibility prediction method using a prediction model constructed by the inventor [Figure 9] Hardware configuration diagram of an information processing device used for machine learning [Figure 10] 1 is a flowchart showing a procedure of a learning process executed by a CPU of an information processing device. [Figure 11] A hardware configuration diagram of an information processing device having a replicated prediction model. [Figure 12] A flowchart showing the procedure of a compatibility prediction process executed by a CPU of an information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Hereinafter, the embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters or duplicate explanation of substantially the same configuration may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the inventor provides the accompanying drawings and the following explanation so that those skilled in the art can fully understand the present disclosure, and the subject matter described in the claims is not limited by them.
[0011] (Findings Obtained by the Inventor) From the results of various research studies accumulated so far, it is known that resting brain function contains information about individual differences. In other words, it has become clear that brain function acquired at rest can be used to predict the brain activity during the individual's later actions. On the other hand, there are also research reports that show that the "compatibility" between two people can be determined by individual differences and behavioral tendencies.
[0012] The inventor hypothesized that by looking at the relationship between the brain functions of two people, it would be possible to predict the psychological phenomena that will occur between the two people, in other words, to predict the compatibility between the two people. The inventor then confirmed that the hypothesis was correct through experiments and verification, and concluded that the information indicating the brain functions of the two people contains information regarding the compatibility between the two people, and that by using this information, it is possible to distinguish whether the "compatibility" between the two people is good or bad. The following is a detailed explanation.
[0013] The inventor conducted a speed dating experiment with multiple men and women. "Speed dating" is a party where men and women who want to get married or date gather together and try to form couples. In this experiment, there were 43 participants in total, 22 men and 21 women, with an average age of 20.3 years old.
[0014] Figure 1 is a schematic diagram showing the speed dating venue. The procedure for the speed dating experiment is as follows. First, the women, Wa to Wu, are seated, and then the men, Ma to Mu, are seated facing each of the women. Then, each pair of men and women talk freely for three minutes. The conversation ends at the moderator's signal, and then, as shown by the arrow, the men all move their seats at once and sit down facing the woman next to them. The above process is repeated until all conversations with the opposite sex are over. After the conversations are over, both male and female participants are asked whether they would like to talk to the opposite sex again, and they answer the questions.
[0015] FIG. 2A shows an example of the results of tallying up answers to questions. Example (a) shows that both the man Ma and the woman Wa wanted to talk to each other. Example (b) shows that the man Mb wanted to talk to the woman Wa again, but the woman Wa did not want to talk to the man Mb again. In this experiment, the case (a) was judged to be "good compatibility," and the case (b) was judged to be "not very compatible."
[0016] Figure 2B shows an example of the relationship between each pair of men and women and their compatibility. From the results of the responses, information was obtained for all pairs of men and women indicating their compatibility, with either "1" indicating "good compatibility" or "0" indicating "not very good compatibility."
[0017] On a separate day before the speed dating experiment, each participant underwent functional magnetic resonance imaging (fMRI) imaging at rest. fMRI is a method of visualizing hemodynamic responses associated with brain activity in humans and animals using MRI (magnetic resonance imaging). Images were taken continuously for 6 minutes. To maintain a resting state, each participant was instructed to do nothing while looking at the ten o'clock mark displayed on the display screen of the fMRI device.
[0018] FIG. 3 shows an example of an fMRI image 100 of a participant's brain. The brain 110 includes multiple brain regions. For example, 116 brain regions are predefined, and each brain region is associated with one of the multiple functions of the brain, such as vision and somatosensation. A time series waveform 102 of each brain region can be obtained from successively captured fMRI images. The time series waveform is obtained as time series digital data (hereinafter also referred to as "time series data 102"). Note that in this specification, the time series data 102 is not data of time series waveforms directly obtained from fMRI images, but data obtained by previously performing noise removal processing on the directly obtained time series waveform data. Noise removal processing is not essential.
[0019] The inventor performed band-pass processing on the obtained time series data 102 and divided the time series data 102 into four frequency bands. In other words, the time series data 102 obtained from each brain region was extracted by dividing it into four frequency bands.
[0020] Figure 4 shows four frequency bands and the waveforms belonging to each frequency band. The four frequency bands are (a) 0.015-0.030 Hz, (b) 0.030-0.059 Hz, (c) 0.059-0.118 Hz, and (d) 0.118-0.216 Hz. The reason for adopting the frequency band of 0.015-0.216 Hz overall is that it was thought that frequencies lower than 0.015 Hz did not contain any information that could be considered meaningful. For the upper limit of 0.216 Hz, the upper limit of the frequency of the data acquired by the fMRI device used in this experiment was adopted. The reason for dividing the frequency bands into four is that recent research has found that these four frequency bands are related to brain functions. However, the specific relationship between each frequency band and each brain function has not yet been found.
[0021] The inventors targeted each individual's signal waveforms, extracted signals from 116 predefined brain regions for each frequency band, and assessed functional connectivity across all combinations of brain regions.
[0022] 5 shows an example of a procedure for calculating a difference feature vector for evaluating functional connectivity. As described above, in this disclosure, it is assumed that there are 116 brain regions, but for generalization, it is assumed that there are M regions (M: an integer of 2 or more).
[0023] Table 1 below shows the functional connectivity between certain human brain regions in the form of an M × M matrix. The frequency bands are any of (a) to (d) above. [Table 1]
[0024] The (i,j) and (j,i) elements of the matrix shown in Table 1 are Pearson correlations that indicate the degree of functional connectivity between the i-th brain region and the j-th brain region. In other words, the matrix shown in Table 1 is a correlation matrix. Therefore, the upper and lower triangular elements of the matrix shown in Table 1 are in a transpose relationship with each other. Note that since Pearson correlation is well known, an example of the calculation formula for the correlation coefficient will be omitted.
[0025] Correlation matrix 120 showing the functional connectivity between the brain regions of male Ma shown in Fig. 5 is an image in which each element of the matrix is colored according to the magnitude of its value for convenience of visualization, as shown in Table 1. Correlation matrix 122 showing the functional connectivity between the brain regions of female Wa shown in Fig. 5 is similar.
[0026] In the present disclosure, the compatibility between the two people is evaluated using the correlation matrix 120 of the male Ma and the correlation matrix 122 of the female Wa. Specifically, first, the absolute value of the difference between the correlation matrix 120 of the male Ma and the correlation matrix 122 of the female Wa is calculated. The obtained difference matrix 124 represents the magnitude of the difference between each component of the correlation matrix 120 of the male Ma and each component of the correlation matrix 122 of the female Wa. The upper and lower triangular components of the difference matrix 124 are also mutually transposed.
[0027] In the present disclosure, a feature vector 126 is generated from the difference matrix 124. The feature vector 126 is a row vector obtained by arranging the upper triangular components or the lower triangular components of the difference matrix 124 in the row direction. The feature vector 126 indicates the degree of difference between the functional connectivity between the brain regions of a male Ma and the functional connectivity between the brain regions of a female Wa.
[0028] The present inventors have presumed that the feature vector 126 contains information on the compatibility between the man Ma and the woman Wa. Therefore, the present inventors have decided to use machine learning to predict the compatibility between men and women.
[0029] FIG. 6 shows a process flow during learning of machine learning according to the present disclosure. Machine learning is performed using a computer system 10. In the present disclosure, machine learning is supervised learning. First, for each of the four frequency bands, correlation matrices of all males and all females in the speed dating are calculated. In addition, for each frequency band, feature vectors 126 are calculated for all combinations of male participants and female participants by the method shown in FIG. 5.
[0030] Then, the computer system 10 performs machine learning using the feature vectors corresponding to pairs of male and female participants and the compatibility of the pairs acquired by the method shown in Figures 2A and 2B as training data 130. By using such training data 130, information on the compatibility between the man and the woman contained in each feature vector 126 is associated with the compatibility, and the results are verified.
[0031] As a machine learning method, the present inventor adopted sparse logistic regression with elastic net regularization terms. Elastic net is a method that uses lasso regression and ridge regression in combination. Lasso regression is a method that optimizes a model to minimize loss by using a loss function with an L1 regularization term added, and reduces unnecessary features. Ridge regression is a method that optimizes a model to minimize loss by using a loss function with an L2 regularization term added, and avoids overfitting. By using elastic net, it is possible to achieve both the reduction in the dimension of features caused by lasso regression and the avoidance of overfitting caused by ridge regression.
[0032] Equation 1 is the evaluation formula for the elastic net. Equation 2 is the P α This is the formula for (β).
number
number
[0033] The above-mentioned "α" and "λ" are hyperparameters. The inventors adopted three candidates for α, (0.0001, 0.001, 0.01), and three candidates for "λ", (0.15, 0.50, 0.85), and determined "α" and "λ" by changing the combinations of these candidates in the following steps (1) to (4). (1) First, divide all the teaching data into “training data” and “test data.” (2) Build a predictive model using the training data for a certain α and λ. (3) Measure the accuracy of the classifier on the test data for the created predictive model. (4) Based on the obtained accuracy, the values of α and λ that give the highest accuracy of the classifier are adopted as the best combination.
[0034] The accuracy of the classifier was determined by 10-fold cross-validation (steps (1) to (3) above). In 10-fold cross-validation, the training data set is divided into 10 parts. Nine of these data sets are used for training, and the remaining one is used for evaluation. Then, the training data set and the evaluation data set are changed and repeated 10 times. For example, 10 cross-validations yield 10 values indicating accuracy. The average value of these values was used as the value indicating accuracy when using that training data (step (4) above).
[0035] FIG. 7A shows the distribution of the actual accuracy compared to the random accuracy, that is, the accuracy distance. The horizontal axis of FIG. 7A shows the accuracy distance from the chance level when the chance level is set to "0.00". The random accuracy is obtained by randomizing the teacher data and then constructing a classifier each time by the above-mentioned process. Here, "randomizing the teacher data" means setting the compatibility label randomly. Originally, the feature vector obtained from a male-female pair with good compatibility is associated with the compatibility label "1" indicating good compatibility, and teacher data is generated. In addition, the feature vector obtained from each male-female pair that cannot be said to have good compatibility is associated with the compatibility label "0" indicating that the compatibility cannot be said to be good, and teacher data is generated. By "randomizing the teacher data", the feature vector and the compatibility label are randomly associated. The inventor attempted to find a significant difference between the actual accuracy and the random accuracy by generating teacher data that was randomly associated separately from the teacher data that was correctly associated. In the experiment, 1,000 random accuracy results were compared with the actual accuracy, and a significant classification result was determined when the probability that the accuracy distance exceeded the chance level, i.e., the probability that the actual accuracy was higher than the random accuracy, was 95% or higher.
[0036] When examined, two pairs of high frequency bands (0.059Hz-0.118 and 0.118Hz-0.216Hz) showed significant discrimination results. This result means that the functional connections of these frequency bands contain information about compatibility. Hereinafter, 0.059Hz-0.118 and 0.118Hz-0.216Hz are referred to as "frequency bands containing information about compatibility". Note that the "frequency bands containing information about compatibility" do not always need to include both frequency bands, and may be either 0.059Hz-0.118 or 0.118Hz-0.216Hz. Furthermore, the "frequency bands containing information about compatibility" may also include a part of the frequency bands 0.059Hz-0.118Hz or 0.118Hz-0.216Hz.
[0037] FIG. 7B shows the results of analyzing the validity of the prediction results of compatibility for the frequency band containing information on compatibility. (a) and (b) show the results of validity analysis for the frequency band of 0.059 Hz to 0.118, and (c) and (d) show the results of validity analysis for the frequency band of 0.118 Hz to 0.216 Hz. On the circumference, brain networks related to 116 brain regions classified by brain function are assigned. Specifically, they are the visual network, somatosensory network, switching network, limbic system, effective control network, default mode network, cerebellum, etc. Each brain network is further subdivided according to function. However, the detailed contents of these are not mentioned in this specification. The purpose is to see the relationship of functional connections of the brain networks.
[0038] Figure 7B (a)-(d) show the relationship of functional connectivity of brain networks based on the feature vectors (or difference matrices) of two people predicted to have good compatibility. The smaller the component of the difference matrix, the more similar the strength of the functional connectivity of the brain network corresponding to that component, i.e., the correlation value of the time-series data between brain regions, is. Conversely, the larger the component of the difference matrix, the more different the strength of the functional connectivity of the brain network corresponding to that component is. (a) and (c) show that the more similar they are, the better the compatibility, and (b) and (d) show that the more different they are, the better the compatibility.
[0039] Along the circumference of (a) to (d) in Figure 7B, the network locations corresponding to some brain functions are labeled with labels indicating the network. The relationship between each label and the brain function is shown below. Vis: Vision SomMot: Somatosensory Cerebellum: Cerebellum (connection with other networks) Limbic: Limbic system (involved in emotions, learning, etc.) Default: Default mode (involved in internal attention and sociality)
[0040] According to Figure 7B, the contributing connections of the networks corresponding to visual and sensorimotor functions (Vis, SomMot) are relatively sparse, while the contributing connections of the networks corresponding to higher brain functions that are associated with personality and behavioral tendencies (Cerebellum, Limbic, Default) are relatively dense. From this, we can infer the following facts: That is, the way in which visual and somatosensory processing takes place in the brain does not contribute much to compatibility. On the other hand, the way in which processing takes place in the brain that is related to personality and behavioral tendencies contributes to compatibility.
[0041] Based on these results, the inventors concluded that it is possible to predict the compatibility between two people by using the functional connections of the two sets of frequency bands described above. In addition, they concluded that it is possible to predict the compatibility between two people, a man and a woman, by constructing a prediction model through machine learning as follows.
[0042] (a) Time series waveforms of each brain region of male and female participants were obtained for each frequency band, and correlation matrices (functional connectivity) were calculated. (b) A feature vector indicating the difference in functional connectivity between brain regions between two people who are known to have good compatibility, calculated using each correlation matrix, is combined with a first compatibility label indicating good compatibility (e.g., "1"). (c) A feature vector indicating the difference in functional connectivity between brain regions between two people who are known in advance to not be compatible, calculated using each correlation matrix, is combined with a second compatibility label (e.g., "0") indicating that they are not compatible. (d) Using the combinations obtained in (b) and (c) above as training data, machine learning is performed to construct a predictive model.
[0043] 8 is a schematic diagram of a compatibility prediction method using a prediction model constructed by the present inventors. The prediction model that has been trained by the above method is constructed in the information processing device 10.
[0044] First, correlation matrices are calculated from the time-series waveforms of each brain region of a man and a woman for which compatibility is to be predicted, and a feature vector 132 is calculated from the difference in absolute values of these correlation matrices. The calculated feature vector is input to the information processing device 10. When the information processing device 10 inputs the feature vector 132 to a prediction model, a value indicating either good compatibility or not good compatibility is output as an output from the prediction model. For example, the output from the prediction model is a compatibility label "1" indicating good compatibility, or a compatibility label "0" indicating not good compatibility.
[0045] Since compatibility labels "1" or "0" are used as training data, binary compatibility labels "1" or "0" are output. However, if the compatibility labels are, for example, four values as training data, it is also possible to output one of the four values from the prediction model. The "four values" here refer to four numerical values (compatibility labels) corresponding to, for example, "very good compatibility," "good compatibility," "bad compatibility," and "very bad compatibility." Furthermore, five compatibility labels including "neither good nor bad compatibility" may be used. The number of labels may be set arbitrarily.
[0046] By performing machine learning according to the type of compatibility label used in the teacher data, the number of types of compatibility output from the prediction model can be three or more. The type of compatibility is determined relatively according to the degree of compatibility. When the number of types of compatibility used as teacher data is three or more, the prediction model may output a label indicating the relatively best compatibility, a label indicating the relatively worst compatibility, and one or more labels relatively between them. In the present disclosure, even if the number of compatibility labels at the time of learning and prediction is two or three or more, at least a label corresponding to the relatively best compatibility (or a first compatibility label indicating good compatibility) and a label corresponding to the relatively worst compatibility (or a second compatibility label indicating that compatibility is not good) are included in the teacher data and output as a prediction result.
[0047] Furthermore, the present inventor obtained the above findings through speed dating as described above, but the above findings are not limited to showing the compatibility between men and women. As long as the prediction is based on the time series waveform of the brain region, the prediction model is believed to be capable of predicting compatibility between people other than between men and women. For example, it is strongly presumed that the compatibility between people of the same sex, such as a boss and a subordinate in a company organization, a teacher and a student in a school organization, a medical professional and a patient, a store clerk and a customer, and the compatibility between people with relatively high and low social or organizational positions can also be appropriately determined.
[0048] Below, a hardware configuration for constructing a prediction model based on the above findings and a compatibility prediction process using the constructed prediction model will be described. 9 is a hardware configuration diagram of an information processing device 10 used when performing machine learning. The information processing device 10 may be, for example, a PC or a server computer. The information processing device 10 includes a CPU 21, a communication interface (I / F) 22, and a storage device 23.
[0049] The CPU 21 is an example of an arithmetic circuit of the information processing device in this embodiment. The CPU 21 executes a control program 26 stored in the storage device 23 to realize predetermined functions including learning and executing a prediction model 27. The information processing device 10 realizes the functions as the information processing device of this embodiment by the CPU 21 executing the control program 26. The control program 26 is an example of a computer program in this embodiment. Note that the arithmetic circuit configured as the CPU 21 in this embodiment may be realized by various processors such as an MPU or a GPU, or may be configured by one or more processors.
[0050] The communication interface 22 is, for example, an Ethernet (registered trademark) communication terminal or a USB (registered trademark) terminal. Alternatively, the communication interface 22 is a communication circuit that performs communication in compliance with standards such as IEEE802.11, 4G, or 5G. The communication interface 22 can be connected to a communication network such as an intranet or the Internet. Furthermore, the information processing device 10 may directly communicate with other devices via the communication interface 22, or may communicate via an access point.
[0051] The storage device 23 is a storage medium that stores computer programs and data necessary for operating the information processing device 10. The storage device 23 may be, for example, a hard disk drive (HDD) or a solid state drive (SSD) that is a semiconductor storage device. The storage device 23 may include a temporary storage element configured by a RAM such as a DRAM or an SRAM, and may function as a working area for the CPU 21. The storage device 23 stores the control program 26 executed by the CPU 21, and stores the prediction model 27 after the prediction model 27 is constructed.
[0052] 10 is a flowchart showing the procedure of the learning process executed by the CPU 21 of the information processing device 10. Also refer to FIG.
[0053] In step S11, the CPU 21 acquires the teacher data 130 (FIG. 6) via the communication interface 22. As shown in FIG. 6, the teacher data 130 is a set of a feature vector and a compatibility label.
[0054] In step S12, the CPU 21 executes machine learning of the prediction model. In this embodiment, the machine learning is executed using sparse logistic regression with an elastic net regularization term.
[0055] In step S13, the CPU 21 adjusts the hyperparameters. In step S14, the CPU 21 generates the prediction model 27. As described in relation to Equation 1 and Equation 2, the prediction model 27 is constructed by using "α" and "λ" as hyperparameters and adopting the values of α and λ when the accuracy of the classifier is the highest.
[0056] The prediction model 27 constructed as described above can be used as is to predict the compatibility between any two people by using the information processing device 10. Alternatively, by duplicating the data constituting the prediction model 27 to any information processing device, the compatibility can be predicted in an information processing device other than the information processing device 10 that generated the prediction model 27.
[0057] 11 is a hardware configuration diagram of an information processing device 30 having a replicated prediction model 27. The information processing device 30 may also be, for example, a PC or a server computer. The information processing device 30 includes a CPU 41, an input interface (I / F) 42, a storage device 43, and an output interface (I / F) 44.
[0058] The CPU 41, the input interface 42, and the storage device 43 have the same configurations and functions as the CPU 21, the communication interface 22, and the storage device 23 shown in Fig. 9. Therefore, the above description of the CPU 21, the communication interface 22, and the storage device 23 will be used to explain the CPU 41, the input interface 42, and the storage device 43.
[0059] It should be noted that a prediction model 27 is stored in the storage device 23. The prediction model 27 is constructed and replicated by the information processing device 10, and is stored in the storage device 23. The prediction model 27 may be incorporated as a part of a computer program executed by the CPU 41, for example, or may be provided as data separate from the computer program.
[0060] The output interface 44 is, for example, a video output terminal connected to a display 50 provided outside the information processing device 30. Alternatively, the output interface 44 is a communication terminal or a communication circuit that is connected to a communication network 52 or the like and is capable of performing data communication. When the output interface 44 is a communication terminal or a communication circuit, the specific content thereof is the same as that of the communication interface 22 (FIG. 9). Therefore, the above description of the communication interface 22 is used here.
[0061] 12 is a flowchart showing the procedure of a compatibility prediction process executed by the CPU 41 of the information processing device 30. It is assumed that two people whose compatibility is to be predicted are predetermined, and that time-series waveforms (time-series data) of each brain region of each person are also prepared in advance.
[0062] In step S21, the CPU 41 acquires time-series data of two people via the input interface 42. In step S22, the CPU 41 uses each time-series data to calculate a correlation coefficient for each combination of two brain regions for each person. In step S23, the CPU 41 calculates a feature vector using the correlation matrices (first correlation matrix and second correlation matrix) calculated from the time-series data of the two people. The feature vector is a row vector obtained by arranging, in the row direction, the upper triangular components or the lower triangular components of a difference matrix calculated as the absolute value of the difference between the first correlation matrix and the second correlation matrix.
[0063] In step S24, the CPU 41 inputs the obtained feature vector to the prediction model 27 and executes the processing of the prediction model 27.
[0064] In step S25, the CPU 41 outputs the compatibility between the two people, which is the prediction result, from the prediction model 27. The CPU 41 may output the prediction result as characters or images on the display 50, or may output the prediction result over the communication network 52. In the latter case, for example, each of the two people can check the prediction result using an information processing device such as their own smartphone.
[0065] In the above embodiment, an example has been described in which the prediction model 27 is constructed by machine learning using sparse logistic regression with an elastic net regularization term. In this embodiment, the prediction model 27 may be generated by other machine learning methods. For example, a support vector machine, a random forest, gradient boosting, an artificial neural network, or the like may be used.
[0066] In this specification, an example has been described in which the time-series data is fMRI images captured continuously. However, the use of fMRI images is only one example. Time-series data acquired by other methods can also be used as long as the method can acquire the activity of multiple brain regions. For example, time-series electroencephalogram data (EEG) acquired by an electroencephalograph showing changes in potential caused by electrical activity of the brain can be adopted. Alternatively, time-series magnetic field data (MEG) acquired by a magnetoencephalograph showing changes in magnetic field caused by electrical activity of the brain can be adopted. Furthermore, time-series brain data, which is changes in cerebral blood flow dynamics measured using near-infrared spectroscopy with a NIRS measuring device, can be adopted. At present, fMRI images need to be acquired at a medical institution, but if helmet-type electroencephalographs and the like become widespread, it will be possible for individuals to acquire EEG, and it is thought that it will be easier to acquire time-series data for using the prediction model 27. Note that the data from fMRI images, EEG, MEG, and NIRS are not exclusive, and multiple combinations may be used.
[0067] The above-mentioned flowcharts (FIGS. 10 and 12) may be realized as a computer program executed by a CPU. FIG. 12 may be realized as a computer program incorporating data of the prediction model 27.
[0068] The present disclosure is not limited to the above-described embodiment, and various modifications are possible. In other words, embodiments obtained by combining technical means appropriately modified by a person skilled in the art are also within the scope of the present disclosure. [Industrial Applicability]
[0069] The present disclosure is applicable to an information processing device, a computer program, and a trained prediction model that predict the compatibility between two people. [Explanation of symbols]
[0070] 10, 30 Information processing device 21 CPU 22 Communication Interface (I / F) 23 Storage device 27 Predictive Models 41 CPU 42 Input interface (I / F) 43 Storage device 44 Output interface (I / F) 100 fMRI images of the brain 102 Time series waveform / time series data 110 Brain 120, 122 Correlation matrix 124 Difference matrix 126 Feature Vector 130 Teacher Data
Claims
1. An information processing device for predicting compatibility between people, An arithmetic circuit; A storage device that stores a prediction model; An input interface for acquiring time-series data showing changes in human brain activity; An output interface that outputs the predicted compatibility between people, Equipped with the human brain includes a plurality of brain regions, each of the plurality of brain regions being associated with a respective one of a plurality of functions of the brain; the time-series data includes data obtained from each of the plurality of brain regions and indicating a temporal change in activity of each of the brain regions; The input interface acquires first time series data and second time series data, which are time series data of a first person and a second person, respectively; The arithmetic circuit includes: using each of the first time-series data and the second time-series data to calculate a correlation coefficient indicating a degree of functional connectivity between brain regions for each combination of two brain regions selected from the plurality of brain regions, thereby generating a first correlation matrix which is a set of correlation coefficients between the first human brain regions and a second correlation matrix which is a set of correlation coefficients between the second human brain regions; Calculating a feature vector indicating a difference in functional connectivity between brain regions between the first human and the second human using the first correlation matrix and the second correlation matrix; inputting the feature vector into the prediction model to predict compatibility between the first person and the second person; The prediction model comprises at least A first type of training data is a combination of a feature vector indicating a difference in functional connectivity between brain regions between two people who are known to have good compatibility in advance and a first compatibility label indicating good compatibility; and A second type of training data is a combination of a feature vector showing the difference in functional connectivity between brain regions between two people who are known in advance to not be compatible with each other, and a second compatibility label showing that they are not compatible with each other. These are constructed by performing machine learning using Information processing device.
2. The time series data is Time series waveform data of a predetermined frequency band obtained from a plurality of fMRI images taken by an fMRI device; Time-series electroencephalogram (EEG) data obtained by an electroencephalograph, which shows changes in electrical potential caused by electrical activity in the brain; Time-series magnetic field data obtained by a magnetoencephalograph, which shows changes in the magnetic field caused by electrical activity in the brain, and / or Time-series brain data showing changes in cerebral blood flow dynamics measured using near-infrared spectroscopy with a NIRS measuring device The information processing device according to claim 1 .
3. 3. The information processing device according to claim 2, wherein when the time series data is time series waveform data of the predetermined frequency band, the frequency band includes all or a part of 0.118 Hz to 0.216 Hz, and / or all or a part of 0.059 Hz to 0.118 Hz.
4. There are M brain regions, the first correlation matrix and the second correlation matrix are M×M matrices; The (i, j) and (j, i) elements of the matrix are correlation coefficients indicating the degree of functional connectivity between the i-th brain region and the j-th brain region.
4. The information processing device according to claim 1.
5. The information processing device according to claim 4 , wherein the feature vector is calculated as an absolute value of a difference between the first correlation matrix and the second correlation matrix.
6. The information processing device according to claim 1 , wherein the prediction model is constructed using sparse logistic regression with an elastic net regularization term.
7. the first human being and the second human being are a male and a female; 7. An information processing device according to claim 1, wherein the two people known to have good compatibility used in constructing the predictive model are a man and a woman, and the two people known to have poor compatibility are a man and a woman.
8. the first person and the second person are of the same sex; An information processing device as described in any one of claims 1 to 6, wherein the two people who are known to have good compatibility and who are used to construct the predictive model are of the same sex, and the two people who are known to have poor compatibility are of the same sex.
9. The first person and the second person are people who have a relatively high or low social or organizational status, An information processing device as described in any one of claims 1 to 6, wherein the two people who are known to have a good compatibility and who are used to construct the predictive model are a person who has a relatively high status and a person who has a relatively low status in society or within an organization, and the two people who are known to not have a good compatibility are a person who has a relatively high status and a person who has a relatively low status in society or within an organization.
10. The storage device holds the time series data, 10. The information processing device according to claim 1, wherein the input interface is an input terminal connected to the arithmetic circuit, and acquires the time-series data read from the storage device and inputs the data to the arithmetic circuit.
11. The time series data is collected or stored by an external device; 10. The information processing apparatus according to claim 1, wherein the input interface is a communication terminal that receives the time series data transmitted from the external device via a communication line.
12. The information processing device according to claim 1 , wherein the output interface is an output terminal connected to the arithmetic circuit, and outputs the prediction result output from the arithmetic circuit.
13. The information processing device according to claim 12 , wherein the output interface is a video output terminal.
14. The information processing apparatus according to claim 1 , wherein the output interface is a communication terminal that outputs the prediction result to a communication line.
15. the prediction model is constructed by performing the machine learning using one or more other types of teacher data in addition to the first type of teacher data and the second type of teacher data; The one or more other types of teacher data, An information processing device as described in any one of claims 1 to 14, which is a combination of a feature vector indicating differences in functional connectivity between brain regions between two people whose degree of compatibility, determined relatively in relation to whether they have good compatibility or not, is known in advance, and one or more compatibility labels indicating the degree of compatibility.
16. A computer program that is executed by an arithmetic circuit of an information processing device to cause the information processing device to predict compatibility between people, The information processing device includes: The arithmetic circuit; A storage device that stores a prediction model; An input interface for acquiring time-series data showing changes in human brain activity; An output interface that outputs the predicted compatibility between people, Equipped with the human brain includes a plurality of brain regions, each of the plurality of brain regions being associated with a respective one of a plurality of functions of the brain; the time-series data includes data obtained from each of the plurality of brain regions and indicating a temporal change in activity of each of the brain regions; The computer program causes the arithmetic circuit to acquiring first time series data and second time series data, which are time series data of a first person and a second person, via the input interface; using each of the first time-series data and the second time-series data, a correlation coefficient indicating a degree of functional connection between brain regions is calculated for each combination of two brain regions selected from the plurality of brain regions, thereby generating a first correlation matrix which is a set of correlation coefficients between the first human brain regions and a second correlation matrix which is a set of correlation coefficients between the second human brain regions; calculating a feature vector indicating a difference in functional connectivity between brain regions between the first person and the second person using the first correlation matrix and the second correlation matrix; inputting the feature vector into the prediction model to predict compatibility between the first person and the second person; The prediction model comprises at least A first type of training data is a combination of a feature vector indicating a difference in functional connectivity between brain regions between two people who are known to have good compatibility in advance and a first compatibility label indicating good compatibility; and A second type of training data is a combination of a feature vector showing the difference in functional connectivity between brain regions between two people who are known in advance to not be compatible with each other, and a second compatibility label showing that they are not compatible with each other. A computer program that is constructed by performing machine learning using each of the above.
17. The computer program of claim 16 , further comprising the predictive model.
18. A trained predictive model for predicting compatibility between a first person and a second person, comprising: The parameters of the trained prediction model include at least A first type of training data is a combination of a feature vector indicating a difference in functional connectivity between brain regions between two people who are known to have good compatibility in advance and a first compatibility label indicating good compatibility; and A second type of training data is a combination of a feature vector showing the difference in functional connectivity between brain regions between two people who are known in advance to not be compatible with each other, and a second compatibility label showing that they are not compatible with each other. These are adjusted using machine learning, Computer, A trained prediction model that functions to accept as input a feature vector indicating differences in functional connectivity between brain regions between the first human and the second human, perform a calculation on the input feature vector based on the parameters, and output a prediction result regarding compatibility between the first human and the second human.
19. the human brain includes a plurality of brain regions, each of the plurality of brain regions being associated with a respective one of a plurality of functions of the brain; The feature vector between the two people is acquiring first time series data and second time series data from each of the two people, the first time series data being time series data acquired from each brain region of one of the two people and showing temporal changes in activity of the respective brain regions, and the second time series data being time series data acquired from each brain region of the other of the two people and showing temporal changes in activity of the respective brain regions; calculating a correlation coefficient indicating a degree of functional connectivity between brain regions for each combination of two brain regions selected from the plurality of brain regions using each of the first time-series data and the second time-series data, thereby generating a first correlation matrix which is a set of correlation coefficients between one brain region of the two people and a second correlation matrix which is a set of correlation coefficients between the other brain region of the two people; calculating an absolute value of a difference between the first correlation matrix and the second correlation matrix; The trained prediction model of claim 18, wherein the trained prediction model is calculated by:
Citation Information
Patent Citations
Method and device for measuring congeniality
JP1995021146A
Device and method for calculating matching success degree
JP2010244468A
Information processing device, determination method, and determination program
WO2020218013A1
