Psychological characteristic estimation device, psychological characteristic estimation system, and psychological characteristic estimation method

Abstract

Provided is a psychological characteristic estimation device including: an acquisition unit that acquires measurement data relating to behavior of a subject; a behavioral feature amount calculation unit that calculates, from the measurement data, a first behavioral feature amount and a second behavioral feature amount having lower environmental dependence than the first behavioral feature amount; a standardization unit that performs standardization processing on the first behavioral feature amount; a psychological characteristic calculation unit that calculates a psychological characteristic of the subject using the second behavioral feature amount and the first behavioral feature amount after the standardization processing; and an output unit that outputs a calculation result of the psychological characteristic. Accordingly, the psychological characteristic of the subject can be estimated with high accuracy.

Description

Mental Trait Estimation Device, Mental Trait Estimation System, and Mental Trait Estimation Method

[0001] The present invention generally relates to a technique for estimating mental traits.

[0002] As an example of personal characteristics, there are mental traits (personality). As a technique for estimating mental traits, for example, the technique disclosed in Patent Document 1 is known. The technique disclosed in Patent Document 1 estimates the mental traits of a user based on text data in addition to the user's speech data.

[0003] In addition, as techniques for estimating a person's mental state, the techniques disclosed in Patent Document 2 and Patent Document 3 are known. The technique disclosed in Patent Document 2 estimates the mental state of a user based on the user's vital data. The technique disclosed in Patent Document 3 estimates the mental traits of a student based on how the student takes a class.

[0004] US10,957,306 JP 2022-179438 A JP 2023-061407 A

[0005] In the technique disclosed in Patent Document 1, in addition to answering questions prepared for purposes other than mental trait estimation, the subject has to perform another action such as speaking for mental trait estimation. It is not always possible for the subject to perform such another action, and thus, the mental traits of the subject may not be estimated. Also, there is a problem that it takes a lot of time to create text data as an input.

[0006] In the techniques disclosed in Patent Document 2 and Patent Document 3, although a temporary mental state can be estimated, it is not possible to estimate the mental traits unique to a person.

[0007] From these facts, there is a need for a technique that can estimate the mental traits of a subject without the subject performing a dedicated action for mental trait estimation. For example, if the mental traits of a person can be estimated from video data of an interview response, the burden related to the interview can be greatly reduced.

[0008] When estimating psychological characteristics from video data, a crucial challenge is how to reduce the influence of the environment. Environmental influences include, for example, the distance between the subject and the camera, the angle of the camera relative to the subject, the brightness of the room, and the subject's body shape and build. Since these environment-dependent factors have a low correlation with psychological characteristics, reducing their influence can be expected to improve the accuracy of psychological characteristic estimation.

[0009] To achieve the above objective, one representative psychological characteristic estimation device and psychological characteristic estimation system of the present invention is characterized by comprising: an acquisition unit that acquires measurement data relating to the behavior of a subject; a behavioral feature calculation unit that calculates a first behavioral feature and a second behavioral feature that is less dependent on the environment than the first behavioral feature from the measurement data; a standardization unit that performs a standardization process on the first behavioral feature; a psychological characteristic calculation unit that calculates the psychological characteristics of the subject using the second behavioral feature and the first behavioral feature after the standardization process; and an output unit that outputs the calculation results of the psychological characteristics.

[0010] Furthermore, one representative psychological characteristic estimation system of the present invention comprises a subject terminal that measures the subject's behavior, and a computing device that receives measurement data regarding the subject's behavior from the subject terminal, wherein the computing device comprises a behavioral feature calculation unit that calculates a first behavioral feature and a second behavioral feature that is less dependent on the environment than the first behavioral feature from the measurement data, a standardization unit that performs a standardization process on the first behavioral feature, a psychological characteristic calculation unit that calculates the subject's psychological characteristics using the second behavioral feature and the first behavioral feature after the standardization process, and an output unit that outputs the calculation results of the psychological characteristics.

[0011] Furthermore, one representative psychological characteristic estimation method of the present invention is characterized in that a computing device includes an acquisition step of acquiring measurement data relating to the behavior of a subject; a behavioral feature calculation step of calculating a first behavioral feature and a second behavioral feature that is less dependent on the environment than the first behavioral feature from the measurement data; a standardization step of performing a standardization process on the first behavioral feature; a psychological characteristic calculation step of calculating the psychological characteristics of the subject using the second behavioral feature and the first behavioral feature after the standardization process; and an output step of outputting the calculation results of the psychological characteristics.

[0012] According to the present invention, the psychological characteristics of a subject can be estimated with high accuracy.

[0013] Conceptual diagram of psychological trait estimation in the embodiment. Explanatory diagram of standardization. Configuration diagram of the psychological trait estimation system. Flowchart explaining the learning process of the standardization model. Flowchart explaining the psychological trait estimation process. Explanatory diagram of the behavioral feature standardization database during learning. Explanatory diagram of the behavioral feature standardization database during use. Variations of the behavioral feature standardization database during use. Specific example of the settings screen on the administrator device.

[0014] In the following description, "interface device" may refer to one or more interface devices. These one or more interface devices may be at least one of the following: • An I / O interface device which is one or more I / O (Input / Output) interface devices. An I / O (Input / Output) interface device is an interface device to at least one of the following: an I / O device or a remote display computer. The I / O interface device to the display computer may be a communication interface device. At least one I / O device may be either a user interface device, such as an input device like a keyboard and a pointing device, or an output device like a display device. • A communication interface device which is one or more communication interface devices. These one or more communication interface devices may be one or more identical communication interface devices (e.g., one or more NICs (Network Interface Cards)) or two or more different communication interface devices (e.g., a NIC and an HBA (Host Bus Adapter)).

[0015] Furthermore, in the following explanation, "memory" refers to one or more memory devices, which are examples of one or more storage devices, and may typically be main memory devices. At least one memory device in memory may be a volatile memory device or a non-volatile memory device.

[0016] Furthermore, in the following explanation, "persistent storage device" may refer to one or more persistent storage devices, which are examples of one or more storage devices. Persistent storage devices are typically non-volatile storage devices (e.g., auxiliary storage devices), and specifically may be, for example, HDDs (Hard Disk Drives), SSDs (Solid State Drives), NVME (Non-Volatile Memory Express) drives, or SCMs (Storage Class Memory).

[0017] Furthermore, in the following explanation, "storage device" may refer to at least memory, including both memory and persistent storage.

[0018] Furthermore, in the following explanation, "processor" may refer to one or more processor devices. At least one processor device may typically be a microprocessor device such as a CPU (Central Processing Unit), but it may also be another type of processor device such as a GPU (Graphics Processing Unit). At least one processor device may be single-core or multi-core. At least one processor device may be a processor core. At least one processor device may be a broad-sense processor device such as a circuit that is a collection of gate arrays (e.g., FPGA (Field-Programmable Gate Array), CPLD (Complex Programmable Logic Device), or ASIC (Application Specific Integrated Circuit)) which is defined by a hardware description language that performs some or all of the processing.

[0019] Furthermore, in the following explanation, functions may be described using the expression "yyy section," but a function may be realized by the execution of one or more computer programs by a processor, by one or more hardware circuits (e.g., FPGA or ASIC), or by a combination thereof. When a function is realized by the execution of a program by a processor, the defined processing is carried out using memory devices and / or interface devices as appropriate, so the function may be at least a part of the processor. Processing described with a function as the subject may also be processing performed by the processor or a device having that processor. Programs may be installed from program source. Program source may be, for example, a program distribution computer or a computer-readable storage medium (e.g., a non-temporary storage medium). The description of each function is an example, and multiple functions may be combined into one function, or one function may be divided into multiple functions.

[0020] Furthermore, in the following explanation, when describing similar elements without distinction, a common reference code will be used, and when describing similar elements with distinction, a reference code will be used.

[0021] The embodiments are described below. In the embodiments described below, psychological characteristics are used as an example of person characteristics, and psychological characteristics are estimated.

[0022] Figure 1 is a conceptual diagram of the estimation of psychological characteristics in the embodiment. The psychological characteristics estimation device 100, described later, estimates the psychological characteristics of the subject using a standardized model M1 and a psychological characteristics estimation model M2. The standardized model M1 is model data generated from multiple interview video data D1, and is data from which environment-dependent features have been excluded from the interview video data D1.

[0023] The psychological trait estimation device 100 uses the subject's interview video data D2 and the standardization model M1 to generate standardized behavioral features D3 by standardizing environment-dependent features. Then, it estimates the subject's psychological traits D4 using the standardized behavioral features D3 and the psychological trait estimation model M2.

[0024] Multiple types of features can be calculated about the subject from the interview video data D1 and interview video data D2. The psychological characteristic estimation device 100 performs standardization using the standardization model M1 as needed on the types of features that are highly dependent on the environment among the multiple types of features calculated from the interview video data D2. For example, features that appear in the subject's way of speaking, facial expressions, movements, tone of voice, etc., due to the questions, have a high correlation with the subject's psychological characteristics and low environmental dependence. In contrast, the distance from the subject to the camera, the angle of the subject to the camera, the amount and direction of lighting, and errors in the motion recognition itself are features that have a low correlation with the subject's psychological characteristics and are highly dependent on the environment. In this embodiment, the system uses the first type of feature, which is highly dependent on the environment among the multiple types of features calculated about the subject, after standardization processing for estimation of psychological characteristics, and uses the second type of feature, which is less dependent on the environment, for estimation of psychological characteristics without standardization processing. Furthermore, even for the first feature, conditions are set for standardization processing. If the conditions are met, the feature is standardized and then used to estimate the psychological trait; if the conditions are not met, the feature is used to estimate the psychological trait without standardization processing.

[0025] Figure 2 is an explanatory diagram about standardization. Figure 2(a) is an explanatory diagram of measurement distance. Comparing video data G1 and video data G2, the distance from the subject to the camera is far in video data G1, while the distance from the subject to the camera is close in video data G2. Therefore, when attempting to estimate psychological characteristics from the subject's movements, the movements are evaluated as small in video data G1 and as large in video data G2, resulting in a decrease in the accuracy of estimating psychological characteristics. By correcting for the differences in characteristics caused by such measurement distances, psychological characteristics can be estimated with greater accuracy. Similarly, even when there are differences in the physique of the subjects, the differences in characteristics caused by these differences in physique can be corrected to estimate psychological characteristics with greater accuracy.

[0026] Figure 2(b) shows the time evolution of feature D10. For example, if the subject sets up the camera themselves and records the interview video, the subject will sit down after setting up the camera, adjust their position relative to the camera and their posture, and then answer the interview questions. In other words, the predetermined time range A1 from the start of recording captures the preparation for the interview and is not suitable as a feature of the interview. Furthermore, using excessively large or excessively small values ​​of feature D10 will reduce the accuracy of estimating psychological characteristics. Therefore, it is desirable to exclude the range A2 that exceeds the maximum value threshold and the range A3 that falls below the minimum value threshold from feature D10. Specifically, the psychological characteristics of the subject are estimated using the range A4 obtained by excluding ranges A1 to A3 from feature D10.

[0027] Figure 3 is a diagram of the configuration of the psychological characteristics estimation system. The psychological characteristics estimation system comprises a psychological characteristics estimation device 100, a subject terminal 200, and an administrator device 300. The psychological characteristics estimation device 100 communicates with the subject terminal 200 and the administrator device 300 via a predetermined communication network. The predetermined communication network is, for example, the Internet, a WAN (Wide Area Network), or a LAN (Local Area Network).

[0028] The subject terminal 200 is an information processing terminal for subject P1, such as a personal computer or smartphone. The subject terminal 200 has one or more sensors for measuring subject P1's actions and a display device, the display 203. The one or more sensors are, for example, a camera 201 and a microphone 202. The subject terminal 200 is also equipped with appropriate input devices (not shown). Subject P1 may be, for example, an applicant in an online interview. The subject terminal 200 outputs interview questions to subject P1 via the display 203, etc. Subject P1 answers the questions. The subject terminal 200 captures the subject P1's answers using the camera 201 and microphone 202 to obtain interview video data. That is, the camera 201 and microphone 202 are examples of sensors, and the interview video data is an example of measurement data. The subject terminal 200 transmits the interview video data to the psychological characteristics estimation device 100 as behavioral signals indicating the subject P1's actions.

[0029] The administrator device 300 is an information processing terminal for the administrator, such as a personal computer or smartphone. The administrator may be, for example, an interviewer in an online interview. If the online interview is a self-interview, the interviewer is a virtual robot such as an avatar, and the administrator does not act as an interviewer but rather evaluates the subject P1 based on the estimation results from the psychological characteristics estimation device 100.

[0030] The psychological characteristics estimation device 100 has an input unit 110, an output unit 120, a calculation unit 130, and a storage unit 140. The input unit 110 and the output unit 120 communicate with the subject terminal 200 and the administrator device 300 via a communication network. The storage unit 140 stores the computer program executed by the calculation unit 130 and the data input and output by the calculation unit 130. The calculation unit 130 is a processor and executes the computer program.

[0031] The input unit 110 includes an action signal acquisition unit 111. The action signal acquisition unit 111 receives action signals from the subject terminal 200. Here, the action signals are video data of the interview. The action signal acquisition unit 111 obtains response information, which indicates the content of the answers, and action information, which is information other than the content of the answers, from the video data of the interview. The action signal acquisition unit 111 outputs the response information and action information to the calculation unit 130.

[0032] The calculation unit 130 implements the functions of the answer calculation unit 131, the behavior calculation unit 132, the behavior feature calculation unit 133, the standardization model calculation unit 134, the behavior feature standardization unit 135, the psychological characteristic calculation unit 136, and the output calculation unit 137 by executing a computer program. The storage unit 140 stores the computer program, as well as the answer calculation result 141, the behavior feature calculation database 142, the behavior feature standardization database 143, the psychological characteristic calculation database 144, the psychological characteristic calculation result 145, and the question presentation information 146.

[0033] The answer calculation unit 131 calculates the answer from the answer information and stores it in the answer calculation result 141. For example, the answer calculation unit 131 converts the audio data, which is the answer information, into text data, recognizes the meaning of the text, and calculates the answer.

[0034] The behavior calculation unit 132 calculates the content of the behavior from the behavior information and outputs it to the behavior feature calculation unit 133. For example, the behavior calculation unit 132 recognizes the posture and movements of the subject P1 as the content of the behavior.

[0035] The behavior feature calculation unit 133 calculates features from the behavior recognized by the behavior calculation unit 132 and outputs them as behavior features to the standardization model calculation unit 134 and the behavior feature standardization unit 135. The behavior feature calculation unit 133 calculates behavior features by referring to the behavior feature calculation database 142. The behavior feature calculation database 142 registers the relationship between behavior and features. By referring to the behavior feature calculation database 142, the behavior feature calculation unit 133 can identify the type of feature to be calculated as a behavior feature and the calculation rule for that feature. Features may include the amount of movement for each body part, the direction of movement, and the acceleration of movement. Features may also include speech duration and voice intonation.

[0036] The standardization model calculation unit 134 calculates standard features using multiple behavioral features and registers them as a standardized model in the behavioral feature standardization database 143. The multiple behavioral features used by the standardization model calculation unit 134 may be calculated using multiple behavioral signals received from multiple subject terminals 200 as a training dataset, or they may be calculated from a pre-provided training dataset.

[0037] The behavioral feature standardization unit 135 standardizes the behavioral features using the behavioral features obtained from the behavioral feature calculation unit 133 and the standardization model stored in the behavioral feature standardization database 143. The behavioral feature standardization unit 135 outputs the standardized behavioral features to the psychological characteristic calculation unit 136.

[0038] The psychological characteristics calculation unit 136 uses the standardized behavioral characteristics and the psychological characteristics calculation database 144 to calculate the psychological characteristics of subject P. The psychological characteristics calculation unit 136 stores the calculated psychological characteristics in the psychological characteristics calculation results 145. The psychological characteristics calculation database 144 is a database that stores data used to estimate psychological characteristics (for example, one or more regression equations or other models).

[0039] The output calculation unit 137 reads the interview questions from the question presentation information 146 and transmits them to the subject terminal 200 via the output unit 120. The output calculation unit 137 also reads the psychological characteristics calculation results 145 and transmits them to the administrator device 300 via the output unit 120.

[0040] The question presentation information 146 is information that includes information designed to induce a behavior in the subject P1 for purposes other than estimating psychological characteristics. For example, it may include multiple (or one) questions provided in voice and / or text by running an AI (Artificial Intelligence) or other program as an interviewer, or content in the form of a virtual robot such as an avatar that provides those questions. The content may include text representing the questions or other types of information (e.g., figures). The "questions" provided in this embodiment may be general interview questions and do not necessarily include questions prepared for the purpose of estimating psychological characteristics.

[0041] The calculation unit 130 receives behavioral information indicating the actions (answers to questions) taken by the subject P1 after being persuaded by the provided information, and obtains an answer calculation result 141 and a psychological characteristic calculation result 145. The "actions persuaded by the persuading information" include the designation of the subject's intention and related actions, which are all or some of the actions excluding the designation of the subject's intention. In this embodiment, the designation of the subject's intention is to answer the question (for example, voice input to the question), but the designation of the subject's intention may vary depending on the information provided. In this embodiment, related actions are all or some of the actions from the time the question is provided until the answer is given to the question, but related actions may also vary depending on the designation of the subject's intention, which depends on the information provided. The subject's intention data is data representing the designated subject's intention. The related action data is data representing the related action.

[0042] "Psychological characteristics" may consist of one or more psychological characteristic components, and one or more psychological characteristic components may include at least one of openness, extraversion, conscientiousness, agreeableness, and emotional stability. The calculation unit 130 may also estimate psychological characteristics based on a portion of the subject's intention data. In addition to the estimated psychological characteristics data, the calculation unit 130 may also estimate the personality characteristics of subject P1 (for example, including at least one of name, gender, date of birth, age, motivation, desired occupation, career history, academic performance, and skills in addition to psychological characteristics) based on subject intention data (for example, response data including name and gender).

[0043] Figure 4 is a flowchart illustrating the learning process of the standardized model. The psychological trait estimation device 100 sequentially executes the following steps S101 to S104. Step S101: The behavior signal acquisition unit 111 acquires a training dataset. Then, it proceeds to step S102. Step S102: The behavior calculation unit 132 calculates behavior from the training dataset, and the behavior feature calculation unit 133 calculates behavior features. As a result, multiple data are obtained for each type of behavior feature. Then, it proceeds to step S103.

[0044] Step S103: The normalization model calculation unit 134 calculates standard features from a plurality of behavior features. The standard features are obtained for each type of behavior feature and used as a normalization model. Then, the process proceeds to step S104. Step S104: The normalization model calculation unit 134 registers the normalization model in the behavior feature normalization database and ends the process.

[0045] FIG. 5 is a flowchart for explaining the estimation process of psychological characteristics. The psychological characteristic estimation system sequentially executes the following steps S201 to S213. Step S201: The subject terminal 200 starts measuring the behavior of the subject P1. Specifically, the start of behavior measurement is the start of shooting by the camera 201 and the start of recording by the microphone 202. Then, the process proceeds to step S202.

[0046] Step S202: The output calculation unit 137 visualizes the fact that measurement is being performed, for example, by lighting a predetermined indicator or the like, and guides the subject P1 to take a position and posture suitable for measurement. Then, the process proceeds to step S203.

[0047] Step S203: The output calculation unit 137 reads out a question from the question presentation information 146 and presents the question by displaying it on the display 203. Then, the process proceeds to step S204. Step S204: The answer calculation unit 131 determines whether it is within the set time for the interview. If it is within the interview time (step S204; Yes), the process proceeds to step S205. If it is not within the interview time (step S204; No), the process proceeds to step S207.

[0048] Step S205: The answer calculation unit 131 determines whether the answer to the question has been completed. If not (step S205; No), the process returns to step S204. If it has been completed (step S205; Yes), the process proceeds to step S206. Step S206: The answer calculation unit 131 determines whether there is a next question. If there is a next question (step S206; Yes), the process returns to step S203. If there is no next question (step S206; No), the process proceeds to step S207. Step S207: The subject terminal 200 ends the measurement of behavior. Then, the process proceeds to step S208.

[0049] Step S208: The action calculation unit 132 calculates the content of the action from the action information, and the action feature quantity calculation unit 133 calculates the feature quantity from the action recognized by the action calculation unit 132. Then, it proceeds to step S209. Step S209: The action feature quantity normalization unit 135 refers to the action feature quantity normalization database 143 and determines whether the type of the action feature quantity obtained from the action feature quantity calculation unit 133 is a target for normalization. If it is specified as a target for normalization (step S209; Yes), it proceeds to step S210. If it is not specified as a target for normalization (step S209; No), it proceeds to step S212.

[0050] Step S210: The action feature quantity normalization unit 135 refers to the action feature quantity normalization database 143 and determines whether the value of the action feature quantity obtained from the action feature quantity calculation unit 133 is a value that requires normalization. If it is a value that requires normalization (step S210; Yes), it proceeds to step S211. If it is not a value that requires normalization (step S210; No), it proceeds to step S212.

[0051] Step S211: The action feature quantity normalization unit 135 normalizes the action feature quantity according to the normalization method shown in the action feature quantity normalization database 143. Then, it proceeds to step S212.

[0052] Step S212: The psychological characteristic calculation unit 136 calculates the psychological characteristics of the subject P using the optionally normalized feature quantity and the psychological characteristic calculation database 144. Then, it proceeds to step S213. Step S213: The output calculation unit 137 transmits the psychological characteristics of the subject P to the administrator device 300 via the output unit 120. Then, the process ends.

[0053] FIG. 6 is an explanatory diagram of the action feature quantity normalization database 143 during learning. The action feature quantity normalization database 143 during learning is data that associates normalization items, normalization methods, feature quantities used for normalization, data intervals used for normalization, calculation methods for standard feature quantities, normalization targets, and normalization conditions.

[0054] In Figure 6, "measured distance" is listed as a standardization item, and the standardization method is "conversion to standard features." The feature used for standardization is "distance between eyes," and the data interval used for standardization is "between 60 seconds and 120 seconds." The method for calculating standard features is the mean, the items to be standardized are "mean," "standard deviation," "minimum value," "first quartile," "median," "third quartile," "maximum value," and "initial value," and the standardization condition is "all."

[0055] This example focuses on how the size of a subject's face image changes depending on the measurement distance. It reduces the influence of measurement distance by transforming the image so that the size of the face becomes the average of the training face image data. The value used to indicate face size is "interpupillary distance," because this value shows relatively little individual variation. Furthermore, for "interpupillary distance," the statistical values ​​specified for standardization ("mean," "standard deviation," "minimum value," "first quartile," "median," "third quartile," "maximum value," and "initial value") are used. The first 60 seconds of video data are excluded from the data interval because they may contain preparation or other video content. The data interval is limited to 120 seconds because sufficient data can be obtained in approximately 60 seconds. Since the standardization condition is "all," all data is standardized. For example, if you want to selectively standardize data that meets specific conditions, you can set those conditions in this standardization condition.

[0056] When performing standardization with new training, it is advisable to set multiple options for the features, data intervals, and calculation methods used for standardization, compare the degree of error reduction and correlation coefficient improvement during the training process, and adopt the most effective setting. If the true values ​​of the measured distance and recognition error itself can be learned, it is also useful to set broad search conditions. For example, if the true values ​​of the measured distance are available, it is easy to evaluate the correlation between the measured distance and the behavioral features, and then process them by removing highly correlated features or correcting them to eliminate the correlation.

[0057] Figure 7 is an explanatory diagram of the behavioral feature standardization database 143 during use. The behavioral feature standardization database 143 during training is data that associates standardization items, standardization methods, features used for standardization, standard features, standardization targets, and standardization conditions.

[0058] In Figure 7, "measured distance" is listed as a standardization item, and the standardization method is "conversion to standard features." The feature used for standardization is "distance between eyes," and the standard feature is "10.00." The items to be standardized are "mean," "standard deviation," "minimum value," "first quartile," "median," "third quartile," "maximum value," and "initial value," and the standardization condition is "all."

[0059] This example demonstrates transforming behavioral features so that the binocular distance in the image of subject P1 is "10.00". Furthermore, for "binocular distance," the statistical values ​​specified for standardization ("mean," "standard deviation," "minimum value," "first quartile," "median," "third quartile," "maximum value," and "initial value") are used. Since the standardization condition is "all," all data is standardized.

[0060] Figure 8 shows variations of the behavioral feature standardization database 143 during use. In Figure 8, the behavioral feature standardization database 143 associates the standardization item "measurement distance" with the standardization method "conversion to standard features", the feature used for standardization "distance between eyes", the standard feature "10.00", the standardization target "maximum value", and the standardization condition "all". The measurement distance has a significant impact on various standardization targets, but the impact on the maximum and minimum values ​​is particularly pronounced when the subject is unable to "get closer" to the camera due to an obstruction such as a desk, or unable to "move further away" due to an obstruction such as a wall behind them.

[0061] Furthermore, the behavioral feature standardization database 143 associates the standardization item "measurement orientation (left / right)" with the standardization method "conversion to standard orientation," the feature used for standardization "ratio of distance between each eye and nose," the standard feature "1.00," the target of standardization "median," and the standardization condition "ratio is less than 0.95."

[0062] Similarly, the behavioral feature standardization database 143 associates the standardization item "measurement orientation (up / down)" with the standardization method "conversion to standard orientation," the feature used for standardization "face aspect ratio," the standard feature "0.75," the standardization target "mean value," and the standardization condition "ratio is 0.8 or greater." Measurement orientation (left / right) and measurement orientation (up / down) correspond to the fact that the feature value changes depending on the relationship between the camera and the orientation of the face. Measurement orientation has a significant impact on various standardization targets, but because the field of view is determined by the desk where it is set up and the subject's terminal, there is always a bias, and the impact on the mean and median is particularly significant. Measurement orientation is corrected to the feature value when assumed to be facing forward using affine transformation, etc.

[0063] Furthermore, the behavioral feature standardization database 143 associates the standardization item "room brightness" with the standardization method "low-pass filter," the feature used for standardization "ratio of face position blur to eye position blur," the standard feature "10," the target of standardization "standard deviation," and the standardization condition "face position blur per frame exceeds eye position blur by 1 degree per field of view." When the room brightness is dark, the detection position of the feature changes. In a dark environment, the eyes can be detected relatively accurately, so the position blur of the entire face, etc., becomes larger compared to the eyes, and has a greater impact on the standard deviation. Therefore, it is corrected to the degree of eye position blur.

[0064] Furthermore, the behavioral feature standardization database 143 associates the standardization item "recognition gaps" with the standardization method "replace with neighboring percentiles," the feature used for standardization "amount of facial movement (field of view)," no standard features, the target of standardization "maximum value," and the standardization condition "greater than 12.5 degrees in field of view units." Feature detection positions may jump if the face is turned too far to the side or obscured by hands, etc. Since jumps in detection positions strongly affect the maximum value, the percentile is lowered until it reaches a value that is possible when there are no jumps in detection positions in the interview video data.

[0065] Furthermore, the behavioral feature standardization database 143 associates the standardization item "Poor initial value" with the standardization method "Replace with data from 60 seconds onward," the feature used for standardization "Amount of nose movement (angle of view)," no standard features, the target of standardization "Initial value," and the standardization condition "Correlation with the immediately following 10 values ​​is less than 0.5." When the interviewee is unaware that recording has started (e.g., not yet ready for the interview) or when the interviewee performs preparation work themselves, data from before they adjusted their facial expressions and posture may be included, resulting in inappropriate features at the start of recording. Since the data at the start of recording is strongly influenced by the initial value, if the data near the initial value is unstable, data from a time period when the person is expected to be relatively calm should be used.

[0066] Figure 9 shows a specific example of the settings screen in the administrator device 300. In the settings screen shown in Figure 9, "measurement distance" and "recognition accuracy" are set as standardization items. For "measurement distance," the standardization method is set to "conversion to standard features," the feature used for standardization is set to "distance between eyes," and the standard feature is set to "10.00." For "recognition accuracy," the standardization method is set to "replace with features with small error."

[0067] Furthermore, for standardization item 1, "measured distance," the standardization method, standardization target, and standardization conditions can be set. For the standardization method, for example, calculation methods such as mean, standard deviation, and initial value can be selected. For the standardization target, for example, body parts such as face size, eye size, and mouth size can be selected. For the standardization conditions, it is possible to apply all time data or to exclude arbitrary intervals.

[0068] Furthermore, for training the standardization model, you can choose to perform new training, use historical data, or not perform training at all. When performing standardization without new training, you can also pre-configure the error factors and reduction methods. The settings screen also has buttons to save the entered settings and to apply the entered settings.

[0069] The psychological trait estimation system of this embodiment uses a standardization model that sets standardization methods and conditions to reduce the influence of environment-dependent factors, i.e., non-psychological trait-related factors. The psychological trait estimation system standardizes each behavioral feature of the user using the standardization model. The standardization model eliminates indicators from among multiple evaluation indicators that are relatively more susceptible to error. As an example, the psychological trait estimation system uses a pre-set standardization model to determine whether or not standardization is necessary for each behavioral feature of each user. For example, the psychological trait estimation system uses a pre-set threshold of the standardization model to determine whether or not standardization is necessary for each behavioral feature. Furthermore, the psychological trait estimation system can set (learn) the conditions and parameters of the standardization model by referring to past and current measurement data. The psychological trait estimation system can use the coefficients calculated during standardization (such as the rate of change before and after standardization) together with the behavioral features for estimation.

[0070] Non-psychological trait-related factors include the distance between the sensor and the subject. The psychological trait estimation system can use features such as face size (area), distance between eyes, nose size, eyebrow size, and mouth size. The psychological trait estimation system can use measurement data from a predetermined interval (e.g., 60 to 120 seconds) from the start of measurement. Non-psychological trait-related factors can include calculation errors for behavioral features. For example, if a behavioral feature exceeds a threshold, the psychological trait estimation system will adopt a smaller percentile value as the behavioral feature. For example, it is possible to "lower the percentile value if the change in the behavioral feature exceeds a field of view of 12.5 degrees," "adopt a value from a later time period as the behavioral feature if the behavioral feature exceeds a threshold," or "use measurement data from 5 to 20 seconds after the start of measurement if the initial behavioral feature does not correlate with the group of behavioral features for the following second by 0.3 or more."

[0071] As described above, according to the disclosed psychological characteristic estimation system, the psychological characteristic estimation device 100 includes an acquisition unit (111) that acquires measurement data relating to the subject's behavior, a behavioral feature calculation unit 133 that calculates a first behavioral feature and a second behavioral feature that is less dependent on the environment than the first behavioral feature from the measurement data, a standardization unit (135) that performs standardization processing on the first behavioral feature, a psychological characteristic calculation unit 136 that calculates the subject's psychological characteristics using the second behavioral feature and the first behavioral feature after the standardization processing, and an output unit 120 that outputs the calculation results of the psychological characteristics. With this configuration and operation, the psychological characteristic estimation system and psychological characteristic estimation device can correct errors in the behavioral analysis results (behavioral features) caused by the influence of the measurement environment and behavioral recognition performance in the estimation of psychological characteristics by analyzing behavioral data, accurately capture various behaviors in various environments, and enable general-purpose and highly accurate estimation of psychological characteristics.

[0072] Furthermore, the output unit 120 outputs questions to the subject, the subject's actions are the answers to the questions, and the behavioral feature calculation unit 133 calculates the first behavioral feature and the second behavioral feature from the measurement data, excluding data that depends on the content of the answers. Therefore, the psychological characteristic estimation system and the psychological characteristic estimation device can separate the answers themselves from the characteristics of the actions induced by the answers, and estimate the subject's psychological characteristics.

[0073] Furthermore, the standardization unit standardizes the first behavioral features using a standardization model that sets a standardization method for each type of the first behavioral feature. With this configuration and operation, the psychological characteristic estimation system and psychological characteristic estimation device can perform standardization according to each of the multiple behavioral features and improve the accuracy of psychological characteristic estimation.

[0074] Furthermore, the standardization model sets further standardization conditions for each type of the first behavioral feature, and the standardization unit decides whether or not to perform the standardization process on the first behavioral feature based on whether or not the conditions are met. If there are first behavioral features that have not undergone the standardization process, the psychological characteristic calculation unit calculates the psychological characteristics using those first behavioral features. With this configuration and operation, the psychological characteristic estimation system and the psychological characteristic estimation system can perform standardization on behavioral features as needed and improve the accuracy of psychological characteristic estimation.

[0075] Furthermore, the psychological characteristics estimation system and the psychological characteristics estimation system further include a standardized model calculation unit that calculates the standardized model using multiple first behavioral features obtained from multiple measurement data for learning. With this configuration and operation, the psychological characteristics estimation system and the psychological characteristics estimation system can appropriately generate a standardized model and use it for estimating the psychological characteristics of a subject.

[0076] As an example, the measurement data is video data captured by a camera, the first behavioral feature is a feature affected by the distance from the camera to the subject, and the standardization process standardizes the first behavioral feature based on the image of the subject's face in the video data. As another example, the measurement data is video data captured by a camera, and the standardization process includes a process of excluding a predetermined amount of time from the start of shooting from the video data. As yet another example, the standardization process includes a process of adopting data within a percentile range specified by a standardization model that sets the standardization method. With this configuration and operation, the psychological characteristic estimation system can perform psychological characteristic estimation by appropriately standardizing the feature.

[0077] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are included. For example, the embodiments described above are explained in detail to make the present invention easier to understand, and are not necessarily limited to those having all the configurations described. Furthermore, it is possible to replace or add configurations, not just delete them.

[0078] For example, in the above explanation, at least one of the following may be adopted: • The behavioral features generated from video data may be at least one of the following: eye movements, eyebrow movements, mouth movements, nose movements, and changes in facial color. • The behavioral features generated from video data may be at least one of the following: pupil fluctuations, gaze fluctuations, degree of fixation, and fixation tremors. • The behavioral features generated from video data may be at least one of the following: head movements, body movements, shoulder movements, arm movements, hand movements, and the subject's position relative to the camera's field of view. • The behavioral features generated from video data may be at least one of the following: pulse rate, stress level, and respiratory rate estimated from changes in facial color between video frames in the video data. • When calculating features, "total movement amount," "movement speed," "movement acceleration," "size change amount," "total rotation amount," "rotation speed," "rotation acceleration," "response time," and "(change in response time)" may be used. Features can be obtained from "count," "mean," "standard deviation," "minimum value," "first quartile," "second quartile," "median," "third quartile," "maximum value," and "initial value."

[0079] 100: Psychological characteristics estimation device 110: Input unit 111: Behavioral signal acquisition unit 120: Output unit 130: Calculation unit 131: Answer calculation unit 132: Behavioral calculation unit 133: Behavioral feature calculation unit 134: Standardization model calculation unit 135: Behavioral feature standardization unit 136: Psychological characteristics calculation unit 137: Output calculation unit 140: Memory unit 141: Answer calculation result 142: Behavioral feature calculation database 143: Behavioral feature standardization database 144: Psychological characteristics calculation database 145: Psychological characteristics calculation result 146: Question presentation information 200: Subject terminal 201: Camera 202: Microphone 203: Display 300: Administrator device

Claims

1. A psychological characteristic estimation device comprising: an acquisition unit for acquiring measurement data relating to the behavior of a subject; a behavioral feature calculation unit for calculating a first behavioral feature and a second behavioral feature that is less dependent on the environment than the first behavioral feature from the measurement data; a standardization unit for performing a standardization process on the first behavioral feature; a psychological characteristic calculation unit for calculating the psychological characteristics of the subject using the second behavioral feature and the first behavioral feature after the standardization process; and an output unit for outputting the calculation results of the psychological characteristics.

2. The psychological characteristic estimation device according to claim 1, characterized in that the output unit outputs questions to the subject, the subject's behavior is an answer to the questions, and the behavior feature calculation unit calculates the first behavior feature and the second behavior feature from measurement data excluding data dependent on the content of the answers.

3. The psychological characteristic estimation device according to claim 1, characterized in that the standardization unit standardizes the first behavioral characteristics using a standardization model in which a standardization method is set for each type of the first behavioral characteristics.

4. The psychological characteristic estimation device according to claim 3, wherein the standardization model further sets standardization conditions for each type of the first behavioral feature, the standardization unit determines whether or not to perform the standardization process on the first behavioral feature based on whether or not the conditions are met, and the psychological characteristic calculation unit calculates the psychological characteristic using the first behavioral feature if there is a first behavioral feature that has not undergone the standardization process.

5. The psychological characteristic estimation device according to claim 3, further comprising a standardization model calculation unit that calculates the standardization model using a plurality of first behavioral features obtained from a plurality of measurement data for learning.

6. The psychological characteristic estimation device according to claim 1, characterized in that the measurement data is video data captured by a camera, the first behavioral feature is a feature that is affected by the distance from the camera to the subject, and the standardization process standardizes the first behavioral feature based on the image of the subject's face in the video data.

7. The psychological characteristic estimation device according to claim 1, wherein the measurement data is video data captured by a camera, and the standardization process includes a process of excluding a predetermined amount of time from the start of recording from the video data.

8. The psychological trait estimation device according to claim 1, characterized in that the standardization process includes a process of adopting data within the percentile range specified by a standardization model that sets the standardization method.

9. A psychological characteristic estimation system comprising: a subject terminal for measuring the subject's behavior; and a computing device for receiving measurement data on the subject's behavior from the subject terminal, wherein the computing device comprises: a behavioral feature calculation unit for calculating a first behavioral feature and a second behavioral feature less dependent on the environment than the first behavioral feature from the measurement data; a standardization unit for performing a standardization process on the first behavioral feature; a psychological characteristic calculation unit for calculating the subject's psychological characteristics using the second behavioral feature and the first behavioral feature after the standardization process; and an output unit for outputting the calculation results of the psychological characteristics.

10. A method for estimating psychological characteristics, comprising: an acquisition step in which a computing device acquires measurement data relating to the behavior of a subject; a behavioral feature calculation step in which a first behavioral feature and a second behavioral feature less dependent on the environment than the first behavioral feature are calculated from the measurement data; a standardization step in which a standardization process is performed on the first behavioral feature; a psychological characteristic calculation step in which the psychological characteristics of the subject are calculated using the second behavioral feature and the first behavioral feature after the standardization process; and an output step in which the calculation results of the psychological characteristics are output.

Images