Person characteristics estimation system and method

Abstract

To estimate the psychological characteristics of a subject without requiring the subject to perform a dedicated action for estimation of psychological characteristics.SOLUTION: A system according to the present invention receives, from a subject device, measurement data pertaining to the action performed by a subject and based on measurement by one or more sensors, and generates, from the measurement data, related action data of a related action which is the whole or some of action except the designation of the subject intention. The system acquires one or more action feature amounts, each representing a non-language feature amount, on the basis of related action data regarding each of one or more related actions, and estimates the psychological characteristics of the subject on the basis of the one or more action feature amounts. The system outputs estimate psychological characteristics data that represents the estimated psychological characteristics.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention generally relates to techniques for estimating person characteristics. [Background technology]

[0002] An example of a person characteristic is a psychological characteristic (personality). A known technique for estimating a psychological characteristic is disclosed in, for example, Patent Document 1. The technique disclosed in Patent Document 1 estimates a user's psychological characteristic based on text data in addition to the user's speech data.

[0003] Known techniques for estimating a person's psychological state include those disclosed in Patent Documents 2 and 3. The technique disclosed in Patent Document 2 estimates a user's psychological state based on the user's vital data. The technique disclosed in Patent Document 3 estimates a student's psychological characteristics based on how the student attends class. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] US10,957,306 [Patent Document 2] Japanese Patent Application Publication No. 2022-179438 [Patent Document 3] Japanese Patent Publication No. 2023-061407 Summary of the Invention [Problem to be solved by the invention]

[0005] In the technology disclosed in Patent Document 1, the subject must respond to questions prepared for a purpose other than psychological trait estimation and must also take other actions, such as speaking, to estimate the subject's psychological trait. The subject may not necessarily take such other actions, and therefore the subject's psychological trait may not necessarily be estimated. Another problem is that it takes a significant amount of time to create the text data used as input.

[0006] The techniques disclosed in Patent Documents 2 and 3 can estimate a temporary psychological state, but cannot estimate a person's unique psychological characteristics. [Means for solving the problem]

[0007] The system receives measurement data from a subject device based on measurements by one or more sensors related to behaviors performed by the subject, and generates related behavior data for related behaviors, which are all or part of the behavior excluding the specification of the subject's intention, from the measurement data. The system acquires one or more behavioral features, each of which is a non-linguistic feature, based on the related behavior data for each of the one or more related behaviors, and estimates the psychological characteristics of the subject based on the one or more behavioral features. The system outputs estimated psychological characteristic data, which is data representing the estimated psychological characteristics. [Effects of the Invention]

[0008] According to the present invention, it is possible to estimate a psychological characteristic of a subject without the subject performing a dedicated behavior for estimating the psychological characteristic. [Brief explanation of the drawings]

[0009] [Figure 1] 1 shows an example of the overall configuration of a system according to a first embodiment. [Figure 2] 2 shows data and functions in the entire system according to the first embodiment. [Figure 3] 1 shows an example of the flow of processing performed in the first embodiment. [Figure 4] An example of the relationship between the target, target details, data items, and behavioral features is shown below. [Figure 5] 1 shows an example of people and landmark detection in a camera frame. [Figure 6A] 10 shows an example of behavioral feature distribution related to the head. [Figure 6B] An example of the distribution of behavioral features related to the shoulders is shown. [Figure 7] An example of the relationship between congruency and maximum response time is shown below. [Figure 8] 10 shows data and functions in the entire system according to the second embodiment. [Figure 9] FIG. 10 is an explanatory diagram of an example of outlier determination. [Figure 10] 1 shows an example of a practical application of the personal characteristic estimation system according to the first and second embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0010] In the following description, an "interface apparatus" may refer to one or more interface devices, which may be at least one of the following: An I / O interface device is one or more I / O (Input / Output) interface devices. The I / O (Input / Output) interface devices are interface devices for at least one of an I / O device and a remote display computer. The I / O interface device for the display computer may be a communications interface device. The at least one I / O device may be a user interface device, for example, either an input device such as a keyboard and a pointing device, or an output device such as a display device. A communication interface apparatus that is one or more communication interface devices. The one or more communication interface devices may be one or more homogeneous communication interface devices (e.g., one or more NICs (Network Interface Cards)) or two or more heterogeneous communication interface devices (e.g., an NIC and an HBA (Host Bus Adapter)).

[0011] In the following description, "memory" refers to one or more memory devices, which are an example of one or more storage devices, and may typically be a primary storage device. At least one memory device in the memory may be a volatile memory device or a non-volatile memory device.

[0012] In the following description, a "persistent storage device" may refer to one or more persistent storage devices, which are an example of one or more storage devices. A persistent storage device may typically be a non-volatile storage device (e.g., an auxiliary storage device), and specifically may be, for example, a hard disk drive (HDD), a solid state drive (SSD), a non-volatile memory express (NVME) drive, or a storage class memory (SCM).

[0013] In the following description, the term "storage device" may refer to at least one of memory and persistent storage device.

[0014] Furthermore, in the following description, a "processor" may refer to one or more processor devices. The at least one processor device may typically be a microprocessor device such as a CPU (Central Processing Unit), but may also be another type of processor device such as a GPU (Graphics Processing Unit). The at least one processor device may be a single-core or multi-core. The at least one processor device may also be a processor core. The at least one processor device may also be a processor device in a broader sense, such as a circuit that is a collection of gate arrays written in a hardware description language that performs some or all of the processing (for example, an FPGA (Field-Programmable Gate Array), a CPLD (Complex Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit)).

[0015] In the following description, functions are sometimes described using the expression "yyy unit." However, the functions may be realized by one or more computer programs executed by a processor, by one or more hardware circuits (e.g., FPGAs or ASICs), or by a combination thereof. When a function is realized by a program executed by a processor, the specified processing is performed using a storage device and / or an interface device, etc., as appropriate, and therefore the function may be considered to be at least a part of the processor. Processing described using a function as the subject may be processing performed by a processor or a device having the processor. A program may be installed from a program source. The program source may be, for example, a program distribution computer or a computer-readable storage medium (e.g., a non-transitory storage medium). The description of each function is merely an example; multiple functions may be combined into one function, or one function may be divided into multiple functions.

[0016] In the following description, when elements of the same type are described without distinction, common reference symbols are used, and when elements of the same type are described with distinction, reference symbols are used.

[0017] Several embodiments will be described below. In the following embodiments, psychological characteristics are used as an example of personal characteristics, and the psychological characteristics are estimated. [First embodiment]

[0018] FIG. 1 shows an example of the overall configuration of a system according to the first embodiment.

[0019] The personal characteristic estimation system 100 communicates with the subject device 130 and the administrator device 180 via a communication network 170. The communication network 170 is, for example, the Internet, a wide area network (WAN), or a local area network (LAN).

[0020] The subject device 130 is an information processing terminal of the subject 101, for example, a computer such as a personal computer or a smartphone. The subject device 130 has one or more sensors that measure the behavior of the subject 101, and a display device 112. The one or more sensors are, for example, a camera 102, an input device 111 (for example, a keyboard and a pointing device), and a microphone 11. Instead of or in addition to the input device 111, the display device 112 may be a touch panel. Furthermore, the subject 101 may be, for example, an applicant for an online interview.

[0021] The administrator device 180 is an information processing terminal of the administrator 151, for example, a computer such as a personal computer or a smartphone. The administrator device 180 has an input device 153 and a display device 152. The administrator 151 may be, for example, an interviewer in an online interview. If the online interview is a self-interview, the interviewer is a virtual robot such as an avatar, and the administrator 151 is not an interviewer, but may be a person who evaluates the subject 101 based on the results of estimation by the personal characteristic estimation system 100.

[0022] The person characteristic estimation system 100 includes an interface device 113, a storage device 114, and a computing device 115 connected thereto.

[0023] The interface device 113 communicates with the subject device 130 and the administrator device 180 via a communication network 170. The storage device 114 stores computer programs executed by the arithmetic device 115 and data input and output by the arithmetic device 115. The arithmetic device 115 is a processor and executes computer programs.

[0024] The computing device 115 provides the subject device 130 with provision information, which is information including invitation information that invites the subject 101 to take action for a purpose other than psychological trait estimation. The invitation information may include multiple (or one) questions. Specifically, for example, the invitation information may include multiple (or one) questions provided in voice and / or text by executing an AI (Artificial Intelligence) or other program as an interviewer, or content such as a virtual robot, such as an avatar, that provides the questions. The content may include text representing the questions or other types of information (e.g., graphics). The "questions" provided in this embodiment may be general questions asked in an interview and may not include questions prepared for psychological trait estimation.

[0025] The computing device 115 receives measurement data from one or more sensors via the interface device 113, based on measurements by the one or more sensors relating to an action taken by the subject 101 in response to the invitation information of the provided information.

[0026] The computing device 115 identifies the target person's intention data and related behavior data based on the measurement data. "Behavior invited by the invitation information" includes a designation of the target person's intention and a related behavior, which is all or part of the behavior excluding the designation of the target person's intention. In this embodiment, the designation of the target person's intention is answering a question (for example, typing or voice inputting an answer to the question), but the designation of the target person's intention may vary depending on the information provided. In this embodiment, the related behavior is all or part of the behavior from when a question is provided to when the question is answered, but the related behavior may also vary depending on the designation of the target person's intention depending on the information provided. The target person's intention data is data representing the designated target person's intention. The related behavior data is data representing the related behavior.

[0027] The arithmetic device 115 calculates one or more behavioral features based on the associated behavior data for each of one or more associated behaviors, and estimates the psychological characteristics of the subject 101 based on the one or more behavioral features. Specifically, for example, each time a question is displayed, the subject 101 answers the displayed question, and there is an associated behavior for each pair of question and answer. There is associated behavior data for each of the multiple associated behaviors, and the arithmetic device 115 calculates one or more behavioral features based on the associated behavior data for the multiple associated behaviors, and estimates the psychological characteristics of the subject 101 based on the one or more behavioral features.

[0028] The arithmetic device 115 outputs estimated psychological characteristic data, which is data representing the estimated psychological characteristic. For example, the arithmetic device 115 transmits the estimated psychological characteristic data to the administrator device 180, and the administrator device 180 displays the psychological characteristic represented by the estimated psychological characteristic data on the display device 152. In this way, the administrator 151 knows the estimated psychological characteristic of the subject 101.

[0029] Each of the one or more behavioral features used to estimate the psychological characteristics of the subject 101 may be adjusted information of at least a portion of the provided information and / or a behavioral feature that has been enhanced over time (examples of enhancement will be described in detail later).

[0030] According to this embodiment, it is possible to estimate the psychological trait of the subject 101 without the subject 101 performing any behavior other than the behavior (e.g., answering a question) invited by the invitation information prepared and provided for a purpose other than psychological trait estimation, that is, without the subject 101 performing any behavior dedicated to psychological trait estimation. When one or more behavioral features used to estimate the psychological trait of the subject 101 are enhanced, it is expected that the accuracy of estimating the psychological trait will be improved.

[0031] The personal characteristic estimation system 100 may be a physical computer system (one or more physical computers) as illustrated in FIG. 1, or alternatively, may be a logical computer system (e.g., a cloud computing service) based on a physical computer system.

[0032] Furthermore, the "psychological trait" may be composed of one or more psychological trait components, and the one or more psychological trait components may include at least one of temperament, character, personality, beliefs, values, mood, and emotion. Furthermore, the computing device 115 may estimate the psychological trait based on a portion of the subject intention data. Furthermore, the computing device 115 may estimate the personal traits of the subject 101 (for example, including at least one of name, gender, date of birth, age, motivation, desired occupation, career history, grades, and possessed skills in addition to the psychological traits) based on the subject intention data (for example, response data including name, gender, etc.) in addition to the estimated psychological trait data.

[0033] The "related behavior" may be the behavior leading up to an answer to a question (an example of specifying the subject's intention). For example, even if the answer is the same, the behavior leading up to the answer is influenced by the psychological characteristics of the subject 101. Since related behavior data representing such behavior is used for psychological characteristic estimation, it is expected that the psychological characteristics can be estimated with high accuracy even if there are no questions or subject behaviors dedicated to psychological characteristic estimation.

[0034] This embodiment will be described in detail below.

[0035] 2 shows the data and functions in the entire system according to the first embodiment. In this embodiment, "DB" stands for database. The data does not have to be structured data like a database.

[0036] As described above, the subject device 130 has a sensor group 201 (one or more sensors) and a display device 112, and also has a control unit 202. The control unit 202 is realized by executing a program (for example, an application program) on a calculation unit (not shown) that is a processor of the subject device 130.

[0037] The control unit 202 transmits measurement data based on measurements by each sensor in the sensor group 201 to the personal characteristic estimation system 100. The control unit 202 also outputs information provided by the personal characteristic estimation system 100 to the display device 112 and / or other output devices (e.g., speakers).

[0038] The storage device 114 stores an answer DB 230, an estimation DB 240, a psychological characteristic DB 250, and a provision DB 260. The answer DB 230 is a DB that stores answer data (data representing answers to questions). The estimation DB 240 is a DB that stores data (e.g., one or more models such as regression equations) used to estimate psychological characteristics. The psychological characteristic DB 250 is a DB that stores estimated psychological characteristic data (data representing estimated psychological characteristics). The provision DB 260 is a DB that stores information to be provided (e.g., content itself such as a question) and information related to that information (e.g., metadata such as the brightness of the content).

[0039] The calculation device 115 executes the computer program to realize a response analysis unit 210 that analyzes responses from the subject 101 and a response control unit 220 that controls responses to the subject 101. The response analysis unit 210 has a response extraction unit 211 that extracts responses, a behavioral data generation unit 290 that generates related behavioral data, a behavioral feature generation unit 212 that generates behavioral features, and a psychological feature estimation unit 213 that estimates psychological features.

[0040] Below, an example of the functions realized by the arithmetic unit 115 and the processing performed in this embodiment will be described.

[0041] FIG. 3 shows an example of the flow of processing performed in the first embodiment.

[0042] The response analysis unit 210 receives video data captured by the camera 102 in the sensor group 201 from the control unit 202 of the subject device 130. The response analysis unit 210 starts recording the video (S301). The recorded data (video data) may be at least a part of the measurement data, and is stored in the storage device 114 by the response analysis unit 210. The response control unit 220 sends a notification of the start of recording to the control unit 202 of the subject device 130, and the control unit 202 may output the notification via the display device 112 or another output device.

[0043] The response control unit 220 provides guidance to the target person 101, and the response analysis unit 210 monitors the target person 101 who has received the guidance (S302). Specifically, the invitation information transmitted by the response control unit 220 to the target person device 130 includes information representing guidance to the target person 101. The guidance may be guidance that contributes to enhancing behavioral features. For example, when the response analysis unit 210 determines from the recorded data that the position of the target person 101 is not appropriate with respect to the angle of view of the camera 102 (or regardless of whether the position is appropriate or not), the response control unit 220 may transmit, to the target person device 130, invitation information including information representing guidance for superimposing the position of the target person 101 at an appropriate position with respect to the angle of view of the camera 102. The guide may include, instead of or in addition to the guide for the position relative to the angle of view of the camera 102, other guides that contribute to enhancing the behavioral features, such as guides for increasing the probability of accurately detecting the vocalizations of the subject 101 (for example, guides for adjusting the microphone settings or adjusting the distance between the microphone and the subject 101).

[0044] The behavioral data generation unit 290 of the response analysis unit 210 identifies, during monitoring, from the video recording data, the behavior of the target person 101 in response to the invitation information including information representing such a guide (an example of related behavior), and generates related behavioral data representing the identified behavior. The behavioral feature generation unit 212 generates behavioral features based on the related behavioral data. The generated behavioral features may be stored in the storage device 114.

[0045] Note that the number of behavioral features does not need to be limited to one, and various features can be integrated to estimate psychological characteristics. For example, the related behavioral data identified by the behavioral feature generation unit 212 based on the video recording data may be data representing at least one of head movement (head movement of the subject 101), facial expression (facial expression of the subject 101), eye movement (eye movement of the subject 101), posture (posture of the subject 101), body movement (body movement of the subject 101), sound (sound of the subject 101's vocalization), vital signs (vital signs of the subject 101), time (time taken for the subject 101 to respond), and device operation (operation of the subject device 130 by the subject 101). The behavioral feature generation unit 212 may generate behavioral features from such related behavioral data. This allows the psychological characteristics of the subject 101 to be estimated from at least one of various perspectives.

[0046] An example of the relationship between the target, target details, data items, and behavioral features is shown in Figure 4. The "target" is the target related to the related behavior data, such as a body region or the time involved in the related behavior. The "target details" are details of the "target," such as which body part's movement is extracted as the related behavior or the duration of the related behavior. The "data item" is a data item of the related behavior data, such as the position and size of a body part or the answer time to each question. The "behavior feature" defines the type of related behavior value and the type of value obtained as the behavioral feature based on the related behavior value. The "related behavior value" is a value (value representing the related behavior) identified from the related behavior data. According to the example shown in Figure 4, the related behavior values ​​include "total movement amount," "movement speed," "movement acceleration," "size change amount," "total rotation amount," "rotation speed," "rotation acceleration," "answer time," and "change amount (in answer time)." Examples of behavioral features include "count," "average value," "standard deviation," "minimum value," "first quartile," "second quartile," "median value," "third quartile," "maximum value," and "initial value." Specifically, for example, a related behavior value obtained from related behavior data identified from video recording data is a movement speed related to head movement, and a behavioral feature such as the maximum or minimum value of the head speed of the subject 101 is obtained from a time series of such movement speed. The behavioral feature generation unit 212 may generate behavioral features using a predetermined generation model (e.g., a regression equation or other model) using one or more related behavior values. A different generation model for generating a behavioral feature may be used for each behavioral feature. Furthermore, a generation model for each behavioral feature may be stored in the storage device 114 (e.g., the estimation DB 240) and identified from the storage device 114.

[0047] After the above-mentioned guidance and monitoring (after S302), the response control unit 220 identifies one or more questions to be provided from the provision DB 260, and provides information to be provided including information representing the identified one or more questions to the target user device 130 (S303). The questions may be provided all at once, or may be provided sequentially (the next question is provided after each answer).

[0048] The response analysis unit 210 may set a time limit for answering one or more provided questions and notify the subject 101 of the time limit. For example, there may not necessarily be a time limit for answering, but if a time limit is associated with one or more provided questions in the provision DB 260, the response analysis unit 210 may set a time limit.

[0049] For example, the response analysis unit 210 determines whether or not the time limit has passed (S304). If an answer is received within the time limit (S304: YES, S305: YES), the answer extraction unit 211 of the response analysis unit 210 extracts an answer from the measurement data (data including data representing the answer input via the input device 111) from the subject device 130, and stores the data representing the extracted answer in the storage device 114. In parallel, the behavioral data generation unit 290 generates associated behavioral data from the measurement data, and stores the generated associated behavioral data in the storage device 114 (S306).

[0050] If there is a next question (S307: YES), the process returns to S304. If all questions have been answered and there is no next question (S307: NO), or if the time limit has expired (S304: NO), the response analysis unit 210 ends the recording (S308). The response control unit 220 sends a notification of the end of recording to the control unit 202 of the target person device 130, and the control unit 202 may output the notification via the display device 112 or another output device.

[0051] The answer extraction unit 211 acquires the answer data stored in the storage device 114, and outputs (stores) the acquired answer data in the answer DB 230 (S309). The answer extraction unit 211 may output the acquired answer data to the administrator device 180.

[0052] The behavior feature generation unit 212 calculates a related behavior value for each related behavior from the related behavior data for each related behavior stored in the storage device 114, and generates a plurality of (or one) behavior feature values ​​using the calculated plurality of (or one) related behavior values ​​(S310).

[0053] The psychological characteristic estimation unit 213 estimates the psychological characteristic of the subject 101 using the model represented by the estimation DB 240 based on the generated multiple (or one) behavioral feature amounts, and outputs (stores) the estimated psychological characteristic data to the psychological characteristic DB 250 (S311). The psychological characteristic estimation unit 213 outputs the estimated psychological characteristic data to the administrator device 180 at S311 or after S311.

[0054] Instead of outputting the answer data (an example of subject intention data) and the estimated psychological characteristic data at different times, integrated data of these data may be output. Furthermore, the output destination of the answer data and the output destination of the estimated psychological characteristic data may be the same or different. Furthermore, the estimated psychological characteristic data may be output to the subject device 130 instead of or in addition to the administrator device 180. This allows the subject 101 to know the estimated psychological characteristic of the subject 101 by answering the questions.

[0055] 3, the measurement data from the subject device 130 includes video data representing a video of the subject 101 captured by the camera 102, and the behavior feature amount generation unit 212 identifies related behavior data for each of one or more related behaviors based on the video data (recorded data). Since various related behavior data can be acquired from the video data, it is expected that the estimation accuracy of psychological characteristics will be improved.

[0056] Furthermore, in the process described with reference to FIG. 3, the response control unit 220 may notify the subject 101 of a time limit for answering one or more questions. Generally, in interviews or surveys, an interviewer, supervisor, or instructor can adjust the time while watching the subject 101's situation and guide the subject 101 to complete the survey of personal characteristics within the set time. However, in automated interviews or surveys, such an interviewer is not present (the subject 101 is interacting with a computer, not a human). For this reason, it is practical to proceed with the interview or survey with a time limit set for questions and responses. Such differences in sensitivity to time limits due to personality are manifested as behavioral feature values ​​with different values ​​related to personality, and as a result, improvement in the accuracy of estimating psychological characteristics is expected.

[0057] Furthermore, in the process described with reference to FIG. 3, each of the one or more behavioral features used to estimate the psychological characteristics of the subject 101 may be adjusted information of at least a part of the provided information and / or behavioral features enhanced over time for the behavioral features. For example, providing the above-mentioned guide contributes to enhancing the behavioral features, as this is expected to generate more accurate behavioral features. "Enhancing" a behavioral feature may involve relatively increasing the behavioral feature and / or its weight, and therefore may include, for example, relatively decreasing one or more behavioral features and / or their weights other than the behavioral feature.

[0058] The following example may be adopted as an example of enhancing a behavioral feature. That is, for example, in S311, the psychological characteristic estimation unit 213 may enhance one or more behavioral features in accordance with the elapsed time for each of the behavioral features. The psychological characteristic estimation unit 213 may estimate the psychological characteristic of the subject 101 based on one or more behavioral features including one or more enhanced behavioral features. During an interview or survey, the subject 101's behavioral characteristics may change (e.g., voice gradually becoming quieter or louder, respiratory rate or vital signs convergence speed changing, gestures and hand movements increasing or decreasing, facial expressions changing, etc.) because the subject 101 becomes accustomed to the atmosphere of the interview or survey. Therefore, by using behavioral features enhanced in accordance with the passage of time (e.g., the degree of adaptation in accordance with the passage of time) for estimating the psychological characteristic (e.g., by taking into account the change in the behavioral feature over time), it is expected that the estimation accuracy of the psychological characteristic will be improved. Furthermore, the degree of appearance of the behavioral feature relative to the time limit can also be reflected in the estimation of the psychological feature, which is expected to improve the accuracy of the estimation of the psychological feature. In this paragraph, some or all of the "one or more behavioral features" may be enhanced behavioral features, in other words, the "one or more behavioral features" may be a mixture of enhanced and non-enhanced behavioral features.

[0059] At least one of the following first to third estimation models may be stored as the estimation model stored in the estimation DB 240. All of the estimation models are multiple regression models, but the estimation model stored in the estimation DB 240 may be another model instead of or in addition to at least one of the first to third estimation models. First estimation model: y=∫(a(t)+b1(t)x1(t)+b2(t)x2(t)+b3(t)x3(t)+···+b n (t)x n (t))dt Second estimation model: y=a+b1x1+b2x2+b3x3+···+b n x n , and b p x p =f(c p1[T1] x p[T1] ,c p2[T2] x p[T2] ,···,c pm[Tm] x p[Tm] ) Third estimation model: y=a+b1x1+b2x2+b3x3+···+b n x n

[0060] The variables used in the above estimation are as follows: ·y is the psychological characteristic component (dependent variable). · a is a constant. ·bx is an explanatory variable. ·x (e.g., x1, x2, , x n Each of these is a behavioral feature. ·b (e.g., b1, b2, , b n each of c) and c (e.g., c p1 ,c p2 ,···,c pm Each of the ) is a weighting coefficient. n and m are each an integer greater than or equal to 1. Both n and m may vary depending on the psychological characteristic component. For example, n may be 5 for one psychological characteristic component, and n may be 3 for another psychological characteristic component. The same applies to m. ·t is the elapsed time. ·T(T1, T2, . . . , Tm) is a time slot (an example of elapsed time). ·p is any integer from 1 to n.

[0061] Each of the first to third estimation models may be prepared for each psychological trait component. The same estimation model may be used for all of the multiple psychological trait components (values ​​substituted for the explanatory variables may differ depending on the psychological trait component), or one of the first to third estimation models may be used for a certain psychological trait component, and another one of the first to third estimation models may be used for another psychological trait component. Components such as neuroticism, openness, conscientiousness, extraversion, and agreeableness may be adopted as the multiple psychological trait components that make up the psychological trait.

[0062] The psychological characteristic estimation unit 213 may estimate at least one psychological characteristic component using a first estimation model. The first estimation model includes a multiple regression model for each elapsed time. The multiple regression model for each elapsed time includes n explanatory variables in a one-to-one correspondence with n behavioral feature quantities (n is an integer greater than or equal to 1, and is a value corresponding to the psychological characteristic component corresponding to the multiple regression model), and a weighting coefficient for each of the n explanatory variables determined by the elapsed time. The first estimation model allows for estimation of psychological characteristics from the entire interview time frame while constantly changing the weighting coefficient over time, which is expected to result in highly accurate psychological characteristic estimation. For example, the first elapsed time may be a first time range from the start of the interview to a first time for a certain behavioral feature quantity, and the second elapsed time may be a second time range from k minutes after the start of the interview to a second time for a different behavioral feature quantity. The time intervals of the first elapsed time and the second elapsed time may be continuous, discrete, or overlapping. The start and end of each elapsed time may be dynamically determined based on behavioral features or other factors. Furthermore, since the elapsed time is typically dynamic, the length or number of elapsed times may differ between when the first estimation model is used to estimate a certain psychological characteristic component and when the first estimation model is used to estimate another psychological characteristic component.

[0063] The psychological trait estimation unit 213 may estimate at least one psychological trait component using a second estimation model. The second estimation model includes n multiple regression models (n is an integer equal to or greater than 1) that correspond one-to-one to n behavioral features. Each of the n multiple regression models includes m weighting coefficients (m is an integer equal to or greater than 2) that correspond one-to-one to m time slots (e.g., the time to be evaluated, such as the duration of an interview) and, for each of the m weighting coefficients, an explanatory variable that is a behavioral feature corresponding to the multiple regression model. This enables psychological trait estimation that takes elapsed time into consideration with fewer computational resources (a smaller computational load on the computing device 115) than the first estimation model. Note that, for one psychological trait component, the lengths of the m time slots may be the same or different. Furthermore, the lengths of the m time slots may be common to all psychological trait components or may differ depending on the psychological trait component (in the latter case, it is desirable that the length of each time slot be appropriate for the behavioral feature as the explained variable). In addition, in the estimation using the second estimation model for a certain psychological characteristic component, a certain explanatory variable is b p x p =c p1[T1] x p[T1] +c p2[T2] x p[T2] +···+c pm[Tm] x p[Tm] and different explanatory variables may be used in different equations.

[0064] The psychological trait estimation unit 213 may select whether to use a first estimation model or a second estimation model for estimating at least one psychological trait component for which elapsed time is preferably taken into consideration, based on a model selection policy (e.g., whether to prioritize computational load or estimation accuracy) including a relationship (typically a magnitude relationship) between the computational load (e.g., processor usage rate) of the computing device 115 and a threshold value for that computational load. For example, when the computational load of the computing device 115 is below the threshold, the psychological trait estimation unit 213 may select the first estimation model, which requires more computational resources but is expected to provide higher estimation accuracy, and estimate the psychological trait component using the first estimation model. On the other hand, when the computational load of the computing device 115 is equal to or greater than the threshold, the psychological trait estimation unit 213 may select the second estimation model, which requires fewer computational resources, and estimate the psychological trait component using the second estimation model. In this way, an optimal estimation model can be selected depending on the computational situation, thereby enabling estimation to be performed according to the computational situation.

[0065] In the above description, at least one of the following may be adopted. The behavioral feature generated based on video data may be at least one of eye movement, eyebrow movement, mouth movement, nose movement, and facial color change. The behavioral feature generated based on video data may be at least one of pupil fluctuation, gaze fluctuation, degree of gaze, and fixational eye movement. The behavioral feature generated based on video data may be at least one of head movement, body movement, shoulder movement, arm movement, hand movement, and the position of the subject 101 relative to the angle of view of the camera 102. The behavioral feature generated based on the video data may be at least one of pulse rate, stress level, and respiratory rate estimated from changes in facial color between video frames in the video data.

[0066] The personal characteristic estimation system 100 may be a server system, and multiple client systems may be systems for multiple organizations (e.g., multiple companies) with different interview (face-to-face) purposes, questions, etc. Personal characteristics are estimated using behavioral features as "non-verbal features" based on data on "related behaviors" that do not depend on questions or answers. Therefore, the psychological characteristics of a subject can be estimated without the subject performing any behavior specifically for psychological characteristic estimation.

[0067] In this embodiment, personal characteristic estimation is performed using a machine learning model such as the estimation model described above. The machine learning model is a model that does not depend on questions or answers, specifically, a model using non-verbal features based on related behavioral data. Therefore, although the content of interview questions generally differs from organization to organization, in this embodiment, there is no need to prepare (train) a machine learning model for each organization. Note that the machine learning model may be another type of model, such as a neural network, instead of the regression model described above.

[0068] Furthermore, in a self-interview in which a virtual robot such as an avatar acts as the interviewer and the subject 101 is the interviewee, unlike an interview in which a human is the interviewer, the subject 101 tends to exhibit small behaviors (reactions). For this reason, it can be difficult to accurately estimate the psychological characteristics of the subject 101 from video data (recorded data) of the interview, but this problem is also solved in this embodiment.

[0069] That is, as illustrated in FIG. 5 , the behavior feature generation unit 212 increases the enhancement amount of the behavior feature based on the related behavior data of the head region 520 of the subject 101 appearing in the video (recording) more than the enhancement amount of the behavior feature based on the related behavior data of the region other than the head region 520 of the subject 101. The "enhancement amount" is the sum of the behavior feature or its weight, and may be zero, a positive value, or a negative value. For example, the behavior feature generation unit 212 may increase the weight of the behavior feature based on the related behavior data of the head region 520 of the subject 101 more than the weight of the behavior feature based on the related behavior data of the region other than the head region 520 of the subject 101. The psychological characteristic estimation unit 213 estimates the person characteristic using an estimation model based on the behavior feature and its weight related to the head region 520 and the behavior feature and its weight related to the region other than the head region 520. For each frame in the video data, the "head region" may be a region that includes the entire head of the subject 101, or a region that is detected as the head.

[0070] Specifically, for example, as illustrated in FIG. 5 , the behavioral data generation unit 290 receives measurement data including video data of the subject 101 during an interview, recognizes feature points such as the entire head, eyes, nose, mouth, eyebrows, shoulders, hands, and fingers for each frame (image) of the video data, and generates the positions and sizes of the recognized feature points and / or parts as related behavioral data. Existing technology (e.g., technology that recognizes landmarks in areas such as the head, face, and hands through image recognition processing) may be used to recognize which parts of the subject 101 are in the image. In this way, related behavioral data may be generated for each of the entire head, eyes, nose, etc. Related behavior values ​​such as total movement amount, movement speed, movement acceleration, magnitude change amount, total rotation amount, rotation speed, and rotation acceleration may be obtained for each M frames (M is a natural number) of the video data. As a result, a time series of related behavior values ​​is obtained for each type of related behavior value. The behavior feature generation unit 212 acquires, as behavior features, at least one of the number of detections, average value, standard deviation, minimum value, first quartile, second quartile, median, third quartile, maximum value, and initial value based on the time series of the related behavior values ​​for at least one type of related behavior value such as total behavior amount, behavior speed, behavior acceleration, magnitude change amount, total rotation amount, rotation speed, and rotation acceleration. The acquired behavior feature does not need to be limited to one, and various behavior feature values ​​can be integrated to estimate psychological characteristics.

[0071] The reason why the enhancement amount of the behavioral feature related to the head region 520 is made larger than the enhancement amount of the behavioral feature related to regions other than the head region 520 is as follows. That is, as illustrated in FIG. 6A, the distribution of the behavioral feature obtained from the movement in the head region (for example, the movement of the entire head, eyes, nose, or mouth) is normal with respect to the psychological characteristic component (the horizontal axis corresponds to the psychological characteristic component, and the vertical axis corresponds to the behavioral feature). Specifically, the higher the value of the psychological characteristic component, the larger the behavioral feature tends to be. On the other hand, as illustrated in FIG. 6B, the distribution of the behavioral feature obtained from the movement of a part other than the head region (for example, the shoulder) is not normal with respect to the psychological characteristic component (the horizontal axis corresponds to the psychological characteristic component, and the vertical axis corresponds to the behavioral feature). Furthermore, when a part other than the head region moves relatively large, that movement is likely to become so-called noise, which may reduce the accuracy of estimating the person's characteristics. For this reason, while the movements of the subject 101 tend to be small overall, there is technical significance in increasing the amount of enhancement of behavioral features obtained from movements in the head region compared to the amount of enhancement of behavioral features obtained from movements outside the head region.

[0072] The behavior feature generation unit 212 increases the enhancement amount of the behavior feature indicating how large the subject's 101 behavior is compared to the enhancement amount of the behavior feature indicating how small the subject's 101 behavior is. In use cases where the subject's 101 behavior tends to be small (e.g., self-interviews), noise is likely to be obtained as the related behavior value, making it difficult to obtain a significant behavior feature. However, behavior features in terms of how large the behavior was performed have less noise and are easier to extract a significant behavior feature. In this way, by increasing the enhancement amount of the behavior feature indicating how large the subject's 101 behavior is, it is expected that the estimation accuracy of psychological characteristics will be improved. The behavior feature indicating how large the subject's behavior is is at least one of the mean, standard deviation, median, third quartile, and maximum of the time series of related behavior values ​​(e.g., total movement amount or movement speed). The behavior feature indicating how small the subject's behavior is is at least one of the minimum, first quartile, and second quartile of the time series of related behavior values.

[0073] The psychological trait estimation unit 213 may input the weighted behavioral feature amount into an estimation model to obtain the value of the psychological trait component as an explained variable. An estimation model may be prepared for each psychological trait component. The same estimation model may be used for all of the multiple psychological trait components (the values ​​substituted for the explanatory variables may differ depending on the psychological trait component).

[0074] As a non-linguistic feature (behavioral feature) based on related behavioral data that is independent of questions and answers, a feature related to answer time may be employed as the behavioral feature instead of or in addition to a behavioral feature of active behavior, such as the movements of the subject 101, as shown in FIG. 4 . Specifically, the related behavioral data may include data representing the length of the answer time from when a question is provided (e.g., displayed) to the subject 101 until the question is answered, and the behavioral feature may include a behavioral feature based on the answer time or its change. Such a behavioral feature may employ the average value, standard deviation, or the like of the answer time from when the question is displayed until the answer, or the change in the answer time. According to the example shown in FIG. 7 , the longer the answer time, the higher the harmony. Therefore, an estimation model can be used to estimate at least the harmony level among the psychological characteristic components based on behavioral features such as the average value and standard deviation of the answer time. Furthermore, the answer time may be measured, for example, as the time from when a question is displayed until the completion button is pressed to answer the question. Measuring (acquiring) the answer time is more accurate and less burdensome than recognizing landmarks such as the face or body from video recording data. As described above, a predetermined time constraint, such as a time limit for answering a question, may be imposed on the subject 101, and a behavioral feature related to the answer time may be extracted based on the answer time and the time constraint. [Second embodiment]

[0075] The second embodiment will be described, focusing mainly on the differences from the first embodiment, and explanations of the commonalities with the first embodiment will be omitted or simplified.

[0076] FIG. 8 shows data and functions in the entire system according to the second embodiment.

[0077] As described above, in a self-interview, the behavior of the subject 101 tends to be small, and therefore, outliers (noise) in the time series example of the related behavior value adversely affect the accuracy of the behavior feature. Examples of causes of outliers include failure to recognize facial landmarks (target points) due to external disturbances, image distortion due to system noise caused by communication, or the connection between videos when videos for each question are spliced ​​into one video.

[0078] Therefore, the response analysis unit 210 has an outlier processing unit 800. The outlier processing unit 800 processes outliers of the associated behavior values ​​obtained from the associated behavior data generated by the behavior data generation unit 290. Since behavior features are extracted from a plurality of associated behavior values ​​after outlier processing, improvement in the accuracy of the behavior features is expected.

[0079] The outliers caused by the above example occur in Y consecutive frames (Y is an integer equal to or greater than 2 (e.g., 3 to 5)). Therefore, the outlier processing unit 800 determines whether there is a sudden change in the associated behavior value (signal change) in the Y consecutive frames. If the result of this determination is true, the outlier processing unit 800 corrects the outliers in those frames.

[0080] Specifically, for example, for each measurement window range 903 of the video data, the outlier processing unit 800 determines whether the change in the related behavior value of Y consecutive frames constituting the measurement window range satisfies a condition. If the result of the determination is true, the outlier processing unit 800 detects the related behavior value that satisfies the condition as an outlier and corrects the detected outlier. More specifically, for example, for the measurement window range 903 (e.g., a time range of several consecutive frames), the outlier processing unit 800 estimates the standard deviation of each sample (related behavior value) relative to the median value within the measurement window range 903. Then, the outlier processing unit 800 determines whether there is a sample within the measurement window range 903 that is greater than N times the standard deviation from the median. A sample for which this determination result is true is an outlier sample (related behavior value). If an outlier is present, the outlier processing unit 800 corrects the outlier, for example, replaces the outlier with the median. Note that Y may be any integer value, for example, ±1 to 3. Therefore, the measurement window range 903 may be a range of ±1 to 3 frames. Furthermore, N may be a predetermined value greater than 1, for example, N=10.

[0081] In this way, the presence or absence of an outlier is determined by whether or not a sudden signal change occurs within the measurement window range 903. For this reason, in the example shown in Fig. 9, sample 901 after a sudden change is an outlier, while sample 902, which has a relatively large value but does not correspond to such a change, is not an outlier.

[0082] If a noise frame has a distinctive pattern on the image, the outlier processing unit 800 may identify the outlier frame by image recognition and correct the related activity value obtained from that frame.

[0083] Although several embodiments have been described above, these are merely examples for explaining the present invention, and it is not intended that the scope of the present invention be limited to these embodiments. The present invention can be implemented in various other forms.

[0084] For example, in any of the embodiments, behavioral features are generated from relevant behavioral data of the subject 101 that changes over a limited time such as an interview, and psychological characteristics are estimated based on the generated behavioral features, thereby realizing automated, objective, and highly accurate estimation of psychological characteristics within a limited time.

[0085] Furthermore, at least one of the first and second embodiments may employ the practical application illustrated in FIG. 10 . That is, a personal evaluation system may exist. The personal evaluation system performs a personal evaluation of a subject (e.g., a decision on whether to hire the subject) based on the subject's response data and estimated psychological characteristic data. The personal evaluation system may be a function within the personal characteristic estimation system 100 or a function external to the personal characteristic estimation system 100 (e.g., a physical computer system or a logical computer system such as a cloud computing service). The personal characteristic estimation system 100 may extract, for each subject, related behavioral data for each answer to a question from the measurement data, and output, to the personal evaluation system, answer data representing the answer to each question and estimated psychological characteristic data representing a psychological characteristic estimated based on the related behavioral data for each answer. Such a practical application can increase the feasibility of automating the personal evaluation of each subject by utilizing the technical means of the personal characteristic estimation system 100. [Explanation of symbols]

[0086] 100: Personality trait estimation system

Claims

1. an interface device connected to a subject device, the subject device being a device having one or more sensors; a computing device connected to the interface device; Equipped with the computing device receives measurement data related to an action performed by the subject and based on measurements by the one or more sensors from the subject device through the interface device; The calculation device generates related behavior data of related behaviors, which are all or part of the behaviors excluding the designation of the subject's intention, from the measurement data; the computing device acquires one or more behavioral features, each of which is a non-linguistic feature, based on related behavior data for each of one or more related behaviors, and estimates a psychological characteristic of the subject based on the one or more behavioral features; the arithmetic device outputs estimated psychological characteristic data, which is data representing the estimated psychological characteristic. Personality trait estimation system.

2. the one or more sensors include a camera; the measurement data includes video data that is data representing a video captured by the camera and showing the subject; the computing device recognizes, from the video data, a region of the subject's head itself or a head region that includes the subject's head; the calculation device makes the enhancement amount of the behavior feature calculated for the head region larger than the enhancement amount of the behavior feature calculated for a region other than the head region. The person characteristic estimation system according to claim 1 .

3. the calculation device increases an enhancement amount of a behavior feature indicating how large the behavior of the subject is compared with an enhancement amount of a behavior feature indicating how small the behavior of the subject is. The person characteristic estimation system according to claim 1 .

4. A related action value, which is a value of the related action, is obtained from the related action data of the related action; the behavior feature indicating the magnitude of the subject's behavior is at least one of the mean value, standard deviation, median, third quartile, and maximum value of the time series of related behavior values; the behavior feature indicating how small the subject's behavior is is at least one of the minimum value, the first quartile, and the second quartile of the time series of related behavior values; The person characteristic estimation system according to claim 3 .

5. The related behavior data includes data representing a length of time from when a question is provided to the subject until when the subject answers the question, the one or more behavioral features include a behavioral feature based on the response time or a change therein; The person characteristic estimation system according to claim 1 .

6. the one or more sensors include a camera; the measurement data includes video data that is data representing a video captured by the camera and showing the subject; A related action value, which is a value of the related action, is obtained from the related action data; The behavioral features are obtained from the time series of the related behavioral values, the computing device determines, for each measurement window range of the video data, whether a change in the related action value of consecutive frames constituting the measurement window range satisfies a condition, and if the result of the determination is true, detects the related action value that satisfies the condition as an outlier, and corrects the detected outlier; The person characteristic estimation system according to claim 1 .

7. the computing device estimates the standard deviation of each related behavior value relative to the median value within each measurement window range, determines whether or not a related behavior value greater than N times the standard deviation from the median value is within the measurement window range (N is a predetermined value greater than 1), and if the determination result indicates that there is an outlier that is a true related behavior value, replaces the outlier with the median value; The person characteristic estimation system according to claim 6 .

8. the measurement data is data based on measurements by the one or more sensors relating to behavior performed by the subject in a self-interview, which is an interview in which a virtual robot is an interviewer and the subject is an interviewee; The person characteristic estimation system according to claim 1 .

9. receiving measurement data from the subject device based on measurements by one or more sensors relating to an action performed by the subject; Generate related behavior data of related behaviors, which are all or part of the behaviors excluding the designation of the subject's intention, from the measurement data; acquiring one or more behavioral features, each of which is a non-linguistic feature, based on related behavior data for each of the one or more related behaviors, and estimating a psychological characteristic of the subject based on the one or more behavioral features; outputting estimated psychological characteristic data which is data representing the estimated psychological characteristic; A method for estimating personal characteristics using a computer.

10. receiving measurement data from the subject device based on measurements by one or more sensors relating to an action performed by the subject; Generate related behavior data of related behaviors, which are all or part of the behaviors excluding the designation of the subject's intention, from the measurement data; acquiring one or more behavioral features, each of which is a non-linguistic feature, based on related behavior data for each of the one or more related behaviors, and estimating a psychological characteristic of the subject based on the one or more behavioral features; outputting estimated psychological characteristic data which is data representing the estimated psychological characteristic; A computer program that causes a computer to do something.

Images