User state estimation apparatus and user state sharing system

The user state estimation device and system accurately assess a user's readiness for communication by evaluating current and past behaviors, correcting evaluation values, and enhancing interaction efficiency in remote work environments.

JP2024165687APending Publication Date: 2024-11-28NTT DOCOMO INC
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023082082
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-18
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

Conventional user state estimation technologies inaccurately reflect a user's availability for communication, leading to missed opportunities for interaction due to misjudging the user's readiness to engage after a break or change in activity.

Method used

A user state estimation device and system that includes detection, determination, and correction units to evaluate a user's current behavior and past actions, determining a first evaluation value, and correcting it based on past behavior trends to estimate the user's state accurately.

Benefits of technology

Enhances the accuracy of estimating a user's condition, allowing for more precise determination of their availability for communication, reducing missed opportunities and improving workplace interaction efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024165687000001_ABST
    Figure 2024165687000001_ABST
Patent Text Reader

Abstract

To estimate the state of a user more accurately.SOLUTION: A detection unit 111 detects an action of a user U-1. A determination unit 112 determines which of groups hierarchized based on an evaluation value indicating a state of the user U-1 current action of the user U-1 at the current time belongs to, to determine a first evaluation value for the current action. A correction unit 113 corrects the first evaluation value based on past actions of the user U-1 in a past predetermined period, to determine a second evaluation value. The estimation unit 114 estimates the state of the user U-1 at the current time, on the basis of the second evaluation value.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a user state estimation device and a user state sharing system. [Background technology]

[0002] In recent years, the introduction of remote work has progressed for reasons such as infection control. Remote work offers conveniences such as more flexibility in working hours and the ability to avoid rush hour traffic. On the other hand, it has reduced opportunities for casual conversation, which was previously a natural part of the workplace, and fewer opportunities for communication between colleagues.

[0003] For example, the following Patent Document 1 discloses a technology that determines the user's situation in real time based on information acquired from a device used by the user for an online conference, and changes the display mode of an avatar corresponding to the user. In this technology, when it is determined that the terminal is in operation and the user is not working, the avatar is displayed in a display mode that indicates that the user is not busy. Other users can look at the user's avatar and determine whether or not it is OK to talk to the user or to request work from the user. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2022-095256 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the above-mentioned conventional technology, the behavior of the user at each time is reflected in the display mode of the avatar. However, whether or not it is acceptable to talk to the user does not necessarily match the behavior of the user at that time. For example, if a user who had been taking a break until just before resumes work, it is predicted that the user will not be concerned if someone talks to him / her because the level of concentration on the work is relatively low immediately after resuming the work. In the above-mentioned conventional technology, even immediately after resuming the work, it is determined that "working = cannot talk to" and other users may miss the opportunity to talk to him / her.

[0006] An object of the present invention is to estimate a user's state more accurately. [Means for solving the problem]

[0007] A user state estimation device according to one embodiment of the present invention includes a detection unit that detects a user's behavior, a determination unit that determines a first evaluation value for the current behavior by determining to which of a plurality of groups hierarchized by evaluation values ​​indicating the user's state the current behavior of the user at a current time belongs, a correction unit that determines a second evaluation value by correcting the first evaluation value based on the user's past behavior in a predetermined past period, and an estimation unit that estimates the user's state at the current time based on the second evaluation value.

[0008] A user state sharing system according to one embodiment of the present invention is a user state sharing system comprising a server, a first user terminal, and a second user terminal, wherein the first user terminal comprises a detection unit that detects the behavior of the first user, a determination unit that determines a first evaluation value for the current behavior by determining to which of a plurality of groups hierarchically organized by evaluation values ​​indicating the state of the first user the current behavior of the first user at the current time belongs, a correction unit that determines a second evaluation value by correcting the first evaluation value based on the past behavior of the first user in a predetermined period of the past, an estimation unit that estimates the state of the first user at the current time based on the second evaluation value, and a state information transmission unit that transmits state information indicating the state of the first user to the server, and the server comprises a provision unit that provides the state information to the second user terminal. Effect of the Invention

[0009] According to one aspect of the present invention, the state of a user can be estimated more accurately. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of a system 1 according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing the configuration of a user terminal 10-1. [Diagram 3] FIG. 2 is a block diagram showing the configuration of a server 20. [Figure 4] 13 is a table showing the details of the actions of groups A to C and their evaluation values. [Diagram 5] FIG. 1 is a diagram illustrating a correction method 1. [Figure 6] FIG. 13 is a diagram illustrating a correction method 2. [Figure 7] FIG. 13 is a diagram illustrating a correction method 3. [Figure 8] FIG. 13 is a diagram showing an example of display of other user information. [Figure 9] 13 is a flowchart showing the operation of the processing device 108 of the user terminal 10-1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] A. First embodiment A-1. System configuration FIG. 1 is a block diagram showing a configuration of a system 1 according to an embodiment. System 1 is an example of a user state sharing system. System 1 includes a plurality of user terminals 10 (10-1 to 10-k (k is an integer equal to or greater than 2)) and a server 20. User terminal 10 is an example of a user state estimation device. User terminals 10-1 to 10-k and server 20 are connected via a network N.

[0012] The user terminals 10-1 to 10-k are held by users U (U-1 to Uk), respectively. In this embodiment, the user terminal 10-1 is an example of a first user terminal, and the user U-1 is an example of a first user. The user terminal 10-2 is an example of a second user terminal, and the user U-2 is an example of a second user. The user terminals 10-1 to 10-k are each an information processing terminal, such as a personal computer, a smartphone, or a tablet terminal.

[0013] In this embodiment, the users U-1 to Uk are assumed to be colleagues who work in the same workplace. Each of the users U-1 to Uk works from their own home and does not meet face to face. However, the users U-1 to Uk can converse with each other (including text communication such as chat) using, for example, a messenger application.

[0014] The server 20 is, for example, a network server installed in the workplaces of users U-1 to Uk. In this embodiment, the server 20 acquires, from each of the user terminals 10-1 to 10-k, status information indicating the status of the user U who uses the user terminal 10. The status of the user U is, for example, whether or not a conversation with the user U is possible. The server 20 provides the acquired status information to the other user terminals 10-1 to 10-k. For example, the user U-1 can access the server 20 using the user terminal 10-1 and check whether or not a conversation with the other users U-2 to Uk is possible by referring to the status information.

[0015] A-2. User terminal 10-1 2 is a block diagram showing the configuration of user terminal 10-1. Since the main configuration of user terminals 10-1 to 10-k is the same, the user terminal 10-1 held by user U-1 will be described as an example. User terminal 10-1 includes a display device 101, an input device 102, a communication device 103, an imaging device 104, a microphone 105, a speaker 106, a storage device 107, a processing device 108, and a bus 109 that interconnects these devices.

[0016] The display device 101 is a display device (for example, various display panels such as a liquid crystal display panel or an organic EL display panel) that displays information to the outside. The input device 102 is an input device (for example, a keyboard, a mouse, a switch, a button, or a sensor) that accepts input from the outside. The input device 102 may be a microphone 105, which will be described later. The display device 101 and the input device 102 may be integrated into one (for example, a touch panel).

[0017] The communication device 103 communicates with the server 20 using wireless communication or wired communication. In this embodiment, the communication device 103 has an interface connectable to a network N, and communicates with a communication device 201 (see FIG. 3) of the server 20 via the network N.

[0018] The microphone 105 picks up sounds made by the user U-1 and sounds generated around the user U-1, and generates voice data. The speaker 106 generates sounds based on voice data acquired from other information processing devices (e.g., other user terminals 10-2 to 10-k). The user U-1 uses the microphone 105 and the speaker 106 to converse with his / her coworkers, users U-2 to U-k.

[0019] The imaging device 104 generates imaging data indicating an image captured by capturing an image of a subject. In this embodiment, the subject is a user U-1 who uses the user terminal 10-1. The imaging device 104 is, for example, an in-camera, and a lens is arranged on the outer frame of the display device 101. The imaging device 104 has, for example, an imaging optical system and an imaging element. The imaging optical system is an optical system including at least one imaging lens. For example, the imaging optical system may have various optical elements such as a prism, or may have a zoom lens or a focus lens. The imaging element is, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary MOS) image sensor.

[0020] The storage device 107 is a recording medium readable by the processing device 108. The storage device 107 includes, for example, a nonvolatile memory and a volatile memory. The nonvolatile memory is, for example, a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), and an Electrically Erasable Programmable Read Only Memory (EEPROM). The volatile memory is, for example, a Random Access Memory (RAM). The storage device 107 stores a program PG1. The program PG1 is a program for operating the user terminal 10-1.

[0021] The processing device 108 includes one or more central processing units (CPUs). The one or more CPUs are an example of one or more processors. Each of the processor and the CPU is an example of a computer.

[0022] By executing the program PG1, the processing device 108 functions as a detection unit 111, a determination unit 112, a correction unit 113, an estimation unit 114, a state information transmission unit 115, and a other user information acquisition unit 116. At least some of the functions of the detection unit 111, the determination unit 112, the correction unit 113, the estimation unit 114, the state information transmission unit 115, and the other user information acquisition unit 116 may be configured by circuits such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array).

[0023] The detection unit 111 detects the behavior of the user U-1. The detection unit 111 detects the behavior of the user U-1 by, for example, analyzing an image of the user U-1 captured by the imaging device 104, the usage state of a keyboard and a mouse which are examples of the input device 102, a detection value of a sensor which is an example of the input device 102, a communication state of the communication device 103, and a log of the processing device 108. In addition, when the user terminal 10-1 can be synchronized with a wearable device such as a smart watch, the detection unit 111 may detect the behavior of the user U-1 based on vital data (heart rate, respiratory rate, body temperature, etc.) of the user U-1 acquired by the wearable device. The detection unit 111 detects the behavior of the user U-1 by, for example, behavior analysis using AI (Artificial Intelligence). The detection unit 111 detects the behavior of the user U-1 at a predetermined detection interval. The detection interval is arbitrary, but in this embodiment, the behavior of the user U-1 is detected, for example, every minute.

[0024] The determination unit 112 determines a first evaluation value for the current behavior by determining to which of a plurality of groups hierarchized by evaluation values ​​indicating the state of user U-1 the current behavior of user U-1 at the current time belongs. The current behavior is the behavior of user U-1 at the current time. In this embodiment, the behavior of user U-1 is classified into three groups, group A, group B, and group C. The plurality of groups A to C are hierarchized by evaluation values ​​indicating the state of user U-1. The state of user U-1 is the tolerance for conversation with user U-1. In other words, the evaluation values ​​of groups A to C indicate the tolerance for conversation with user U-1. The evaluation value increases as the tolerance for conversation with user U-1 increases.

[0025] FIG. 4 is a table showing the contents of the actions of groups A to C and their evaluation values. Group A is an action in which user U-1 transmits information. Group A can also be described as an action in which user U-1 outputs his or her own thoughts. Actions classified into group A include, for example, participating in a meeting or conversation, making a phone call, inputting using a keyboard, or writing with a writing implement.

[0026] Group B is the behavior of user U-1 collecting information. Group B can also be described as the behavior of user U-1 inputting information necessary for thinking. Behaviors classified into group B include, for example, operating a mouse or a touch panel, operating a smartphone or a tablet terminal different from the user terminal 10-1, reading a book or a document, etc.

[0027] Group C is behavior other than group A and group B. That is, group C is behavior other than sending or collecting information. Examples of behaviors classified into group C include eating food, drinking beverages, stretching, or doing nothing.

[0028] Comparing groups A to C, the behavior of group A involves conversation with others or is work that requires concentration, so the tolerance of conversation is relatively low. In this embodiment, the evaluation value for the behavior of group A is "1". The behavior of group B does not require as much concentration as group A, but is work that is not preferable to be interrupted midway, so the tolerance of conversation is medium. In this embodiment, the evaluation value for the behavior of group B is "2". The behavior of group C is considered to be a refreshment period between work, so the tolerance of conversation is relatively high. In this embodiment, the evaluation value for the behavior of group C is "3". That is, the multiple groups are hierarchically divided into group A, group B, and group C in order of decreasing evaluation value. The determination unit 112 determines which of group A, group B, and group C the current behavior corresponds to, and determines the evaluation value of the group to which the current behavior corresponds as the first evaluation value.

[0029] The correction unit 113 determines the second evaluation value by correcting the first evaluation value based on the past behavior of the user U-1 in a predetermined period of time in the past. The past behavior is the behavior of the user U-1 in a predetermined period of time in the past. The predetermined period is a period during which the continuity of the behavior of the user U-1 is recognized, and can be, for example, several seconds to several minutes (about 2 seconds to 10 minutes) or about several tens of minutes (about 10 minutes to 30 minutes).

[0030] The correction unit 113 corrects the first evaluation value in consideration of the tendency of the user U-1's behavior from the past to the present. Specific examples of the correction by the correction unit 113 will be described below as correction methods 1 to 3.

[0031] [Correction method 1] 5 is a diagram showing a schematic diagram of correction method 1. n and ni (i is a positive integer) indicate the passage of time, and the larger the value, the later the time. n indicates the current time, and A, B, or C shown in the n column indicates the group to which the current action belongs. ni indicates a past time, and A, B, or C shown in the ni column indicates the group to which the past action belongs. The interval between n-2 and n-1, and the interval between n-1 and n correspond to the detection interval by detection unit 111.

[0032] In the first behavior history, the behavior of group A is performed at n-2 and n-1. Therefore, it is estimated that the behavior of group A has continued at least between n-2 and n-1. Meanwhile, at n, which is the current time, the behavior of group B is performed. As described above, the evaluation value for the behavior of group B is "2". Therefore, the first evaluation value is "2". Meanwhile, since the behavior of group A has continued at least between n-2 and n-1, the correction unit 113 determines the second evaluation value by bringing the first evaluation value closer to that of group A. Specifically, the second evaluation value is determined to be "1", which is the evaluation value of group A.

[0033] In addition to setting the second evaluation value to "1", which is the evaluation value of group A itself, correction unit 113 may set the second evaluation value to a value less than 2 and greater than 1, such as "1.5", to bring the second evaluation value closer to the evaluation value of group A.

[0034] Also, for example, when actions of different groups are performed in n-2 and n-1, that is, when the period of an action belonging to a specific group does not meet a predetermined period, the correction unit 113 does not perform correction. "No correction" may mean that the correction amount of the first evaluation value is set to "0." For example, when actions of group A are performed in n-2 and group B are performed in n-1, and an action of group A is performed at the current time n, both the first evaluation value and the second evaluation value are "1," which is the evaluation value of group A.

[0035] Furthermore, in a correction at a time later than the current time, for example at n+1, the behavior at n (which is past behavior at n+1) is treated as the actual behavior, that is, the behavior of group B.

[0036] In the second behavior history, the behavior of group A is occurring at n-2 and n-1, while the behavior of group C is occurring at the current time, n. Therefore, the first evaluation value is the evaluation value of group C, "1". However, since the behavior of group A continues at least between n-2 and n-1, the correction unit 113 corrects the first evaluation value to the evaluation value of group B, which is one level closer to group A than group C, and determines the second evaluation value to be "2".

[0037] In the third behavior history, group C's behavior is occurring at n-2 and n-1, while group B's behavior is occurring at the current time, n. Therefore, the first evaluation value is group B's evaluation value of "2". However, since group C's behavior continues at least between n-2 and n-1, the correction unit 113 corrects the first evaluation value and determines the second evaluation value to be group C's evaluation value of "3".

[0038] In the fourth behavior history, the behavior of group C is occurring at n-2 and n-1, while the behavior of group A is occurring at the current time, n. Therefore, the first evaluation value is the evaluation value of group A, "1." However, since the behavior of group C continues at least between n-2 and n-1, the correction unit 113 corrects the first evaluation value and determines the second evaluation value to be "2," which is the evaluation value of group B, which is one level closer to group C than group A.

[0039] In this way, in correction method 1, when the multiple groups include a first group and a second group, if the past behavior belongs to the second group and the current behavior belongs to the first group, the correction unit 113 determines the second evaluation value by making the first evaluation value closer to the evaluation value of the second group. For example, in the first behavior history, group B corresponds to the first group and group A corresponds to the second group.

[0040] Here, in the above-mentioned 1st to 4th behavior histories, the behavior of group A or group C is performed in n-2 and n-1. Group A is the behavior with the lowest tolerance for conversation with user U-1 among groups A to C, and group C is the behavior with the highest tolerance for conversation with user U-1 among groups A to C. Therefore, correction by the correction unit 113 is considered to be effective. On the other hand, group B is an intermediate behavior, and correction may not necessarily be effective. Therefore, when the behavior of group B is performed in n-2 and n-1, one of the two methods shown in the behavior histories of 5-1 or 5-2 may be selected.

[0041] The 5-1st behavior history is a method of correction similar to the 1st to 4th behavior histories. That is, while group B's behavior is taking place at n-2 and n-1, group C's behavior is taking place at n, which is the current time. Therefore, the first evaluation value is "3", which is the evaluation value of group C. In this case, the correction unit 113 corrects the first evaluation value and determines the second evaluation value to be "2", which is the evaluation value of group B.

[0042] The 5-2nd behavior history is a method that does not perform correction. That is, at the current time n, group C's behavior is being performed. Therefore, the first evaluation value is the evaluation value of group C, "3." If correction is not performed by correction unit 113 (or if the amount of correction is "0"), the second evaluation value will be "3", the same as the first evaluation value.

[0043] [Correction method 2] 6 is a diagram showing a schematic diagram of correction method 2. In correction method 2, an evaluation value for an action at the current time n is corrected by taking into consideration actions further in the past (n-3) than in correction method 1.

[0044] In the sixth behavior history, the behavior of group C occurred at n-3, group B at n-2, group C at n-1, and group B at n. Therefore, the first evaluation value is "2". If only correction method 1 is applied, no correction is made in the sixth behavior history because the behavior of a specific group has not continued for a specified period of time in the past. Therefore, the second evaluation value is "2", the same as the first evaluation value.

[0045] In contrast, in correction method 2, the behavior that occurred for the longest period among n-3 to n-1 is corrected to be the behavior that occurred for the period from n-3 to n-1. That is, as shown in the 6ath behavior history, the behavior that occurred for the longest period among n-3 to n-1 is the behavior of group C, so it is corrected to be the behavior of group C also in n-2. When correction method 1 is applied to the corrected behavior history, the first evaluation value is corrected and the second evaluation value is determined to be the evaluation value of group C (not shown, but "3").

[0046] Similarly, in the seventh behavior history, the behaviors of group A occurred at n-3, group C at n-2, group A at n-1, and group C at n. Therefore, the first evaluation value is "3." If only correction method 1 is applied, no correction is made in the seventh behavior history because the behavior of a specific group has not continued for a specified period of time in the past. Therefore, the second evaluation value is "3," the same as the first evaluation value.

[0047] In contrast, in correction method 2, as shown in the 7ath behavior history, the behavior of group A that was performed for the longest period from n-3 to n-1 is corrected to be performed also in n-2. When correction method 1 is applied to the corrected behavior history, the first evaluation value is corrected, and the second evaluation value is determined to be the evaluation value of group B ("2" although not shown).

[0048] In this way, in correction method 2, when multiple actions belonging to different groups are performed during a predetermined period, the correction unit 113 corrects the content of the past action based on the duration of each of the multiple actions. More specifically, the correction unit 113 corrects the content of the past action by considering the action with the longest duration (including before the predetermined period) among the multiple actions to have continued for the predetermined period.

[0049] [Correction method 3] FIG. 7 is a diagram showing a schematic diagram of correction method 3. In correction method 3, the evaluation value for the behavior at the current time n is corrected by considering more detailed contents of the behaviors of groups A to C. The correction rule in correction method 3 is set by each of users U-1 to Uk according to their own work contents, for example. The set correction rule is stored in each of user terminals 10-1 to 10-k and used for the correction process in the user terminals 10-1 to 10-k. The example shown in FIG. 7 is the correction rule set by user U-1, and a different correction rule is set in user terminal 10-2 used by user U-2, for example. User terminal 10-1 may be provided with a reception unit that receives the correction rule of correction method 3.

[0050] The set correction rule is also transmitted to the server 20 and is used to grasp the status of the correction process in each of the user terminals 10-1 to 10-k. The correction rule set by many users U (e.g., 60% or more of the users U-1 to Uk) may be applied to the entire system 1. Furthermore, the correction rule for correction method 3 may not be set by each of the users U-1 to Uk, but may be determined by a workplace manager or the like.

[0051] In the eighth behavior history, a keyboard operation, which is an behavior of group A, is performed at n-1, and a mouse operation, which is an behavior of group B, is performed at n. Therefore, the first evaluation value is "2", which is the evaluation value of group B. On the other hand, the correction unit 113 corrects the first evaluation value and determines the second evaluation value to be "1", which is the evaluation value of group A. This is a correction rule set by user U-1, since a mouse operation immediately following a keyboard operation is often part of a series of tasks that includes a keyboard operation.

[0052] In the ninth behavior history, stretching, which is an behavior of group C, is performed at n-1, and keyboard operation, which is an behavior of group A, is performed at n. Therefore, the first evaluation value is "1", which is the evaluation value of group A. On the other hand, the correction unit 113 corrects the first evaluation value and determines the second evaluation value to be "3", which is the evaluation value of group C. This is because after changing one's mood by stretching, it takes a certain period of time to concentrate on work again. Therefore, this is a correction rule set by user U-1, based on the idea that it will not be a problem if someone talks to him / her after he / she starts keyboard operation.

[0053] In the tenth behavioral history, at n-1, drinking a drink, which is a behavior of group C, is performed, and at n, operating a keyboard, which is a behavior of group A, is performed. Therefore, the first evaluation value is "1", which is the evaluation value of group A. In this case, the correction unit 113 does not perform a correction, and the second evaluation value is determined to be "1", which is the same as the first evaluation value. This is a correction rule set by user U-1 based on the idea that, unlike stretching, drinking a drink is performed for an extremely short time while working, and therefore does not affect concentration on work.

[0054] In this way, each of the groups A to C includes a plurality of types of behavior. In the third correction method, the correction unit 113 corrects the first evaluation value based on the relationship between the type of the current behavior and the type of the past behavior.

[0055] The estimation unit 114 estimates the state of user U-1 at the current time based on the second evaluation value. The estimation unit 114 estimates whether or not conversation with user U-1 is possible as the state of user U-1. As described above, in this embodiment, the evaluation value is larger as the tolerance for conversation with user U-1 is higher. Therefore, when the second evaluation value is equal to or greater than a predetermined threshold, the estimation unit 114 estimates that conversation with user U-1 is possible, and when the second evaluation value is less than the threshold, the estimation unit 114 estimates that conversation with user U-1 is not possible.

[0056] For example, the estimation unit 114 estimates that conversation with the user U-1 is possible when the second evaluation value is 3, and estimates that conversation with the user U-1 is not possible when the second evaluation value is less than 3 (2 or 1).

[0057] Furthermore, the estimation unit 114 may estimate in more stages whether or not conversation with the user U-1 is possible. For example, according to the above criteria, if the second evaluation value is less than 3 (2 or 1), it is estimated that conversation with the user U-1 is impossible, but if the second evaluation value is 2, the tolerance for conversation is higher than if the second evaluation value is 1. Therefore, the estimation unit 114 may estimate, for example, that if the second evaluation value is 2, "conversation is impossible except for urgent matters (conversation is possible for urgent matters)" and that if the second evaluation value is 1, "conversation is impossible even for urgent matters."

[0058] Also, for example, when the second evaluation value includes a decimal such as "1.5", the threshold value may be a number including a decimal. For example, the estimation unit 114 may estimate that a conversation with the user U-1 is possible when the second evaluation value is 2.5 or more, and may estimate that a conversation with the user U-1 is not possible when the second evaluation value is less than 2.5.

[0059] The state information transmission unit 115 transmits state information indicating the state of the user U-1 to the server 20. The state information transmission unit 115 transmits the state information to the server 20, for example, every time the estimation unit 114 estimates the state of the user U-1.

[0060] The other user information acquisition unit 116 acquires status information of other users U-2 to Uk other than the user U-1 (hereinafter referred to as "other user information"). The other user information acquisition unit 116 transmits an acquisition request for other user information to the server 20, and acquires the other user information transmitted by the server 20 in response to this. The other user information will be described in detail later.

[0061] A-3. Server 20 3 is a block diagram showing the configuration of the server 20. The server 20 includes a communication device 201, a storage device 202, a processing device 203, and a bus 209 that interconnects these devices.

[0062] The communication device 201 communicates with the user terminals 10-1 to 10-k using wireless communication or wired communication. In this embodiment, the communication device 201 has an interface connectable to a network N, and communicates with the user terminals 10-1 to 10-k via the network N.

[0063] The storage device 202 is a recording medium readable by the processing device 203. The storage device 202 includes, for example, a nonvolatile memory and a volatile memory. The nonvolatile memory is, for example, a ROM, an EPROM, and an EEPROM. The volatile memory is, for example, a RAM. The storage device 202 stores a program PG2. The program PG2 is a program for operating the server 20.

[0064] The processing unit 203 includes one or more CPUs. The one or more CPUs are examples of one or more processors. Each of the processor and the CPU is an example of a computer.

[0065] The processing device 203 executes the program PG2 to function as a state information acquisition unit 211 and a providing unit 212. At least some of the functions of the state information acquisition unit 211 and the providing unit 212 may be configured by circuits such as a DSP, an ASIC, a PLD, and an FPGA.

[0066] The state information acquisition unit 211 acquires state information of each of the users U-1 to Uk from the user terminals 10-1 to 10-k. The state information acquisition unit 211 receives state information transmitted from the state information transmission unit 115 of each of the user terminals 10-1 to 10-k.

[0067] The providing unit 212 provides the status information of each of the users U-1 to Uk to the user terminals 10-1 to 10-k. In other words, the providing unit 212 provides the status information of the user U-1 to a user terminal 10 (for example, the user terminal 10-2) owned by a user U other than the user U-1. The user terminal 10-2 is an example of a second user terminal.

[0068] For example, in response to a request to acquire other user information transmitted from the other user information acquisition unit 116 of the user terminal 10-1, the providing unit 212 transmits status information of users U-2 to 10-k other than the user U-1 (or status information of all users U-1 to 10-k including the user U-1) to the user terminal 10-1. When the other user information acquisition unit 116 of the user terminal 10-1 acquires the status information (other user information) from the providing unit 212, it displays the other user information on the display device 101.

[0069] FIG. 8 is a diagram showing an example of displaying other user information. FIG. 8 shows an example in which the status of user U is displayed in imitation of the seating arrangement in a workplace. Other user information S in FIG. 8 includes a desk image D (D-1) and user images C (C-1 to C-6). Desk image D-1 shows the seat of user U-1, and user image C-1 shows the status information of user U-1. Desk image D-2 shows the seat of user U-2 (Mr. A), and user image C-2 shows the status information of user U-2. Similarly, desk image Dj (j=an integer from 1 to 6) shows the seat of user Uj, and user image Cj shows the status information of user Uj.

[0070] The user image Cj is an icon that resembles a human face. When the user Uj is able to talk (when the evaluation value is 3), the user image Cj is displayed with a smiling face. In FIG. 8, the user image C-1 corresponding to the user U-1 and the user image C-3 corresponding to the user U-3 indicate that they are able to talk. Also, when the user Uj is unable to talk (when the evaluation value is 2 or 1), the user image Cj is displayed with a facial expression other than a smiling face. For example, when conversation is not possible except for urgent matters (when the evaluation value is 2), the user image Cj is displayed with a neutral facial expression. In FIG. 8, the user image C-4 corresponding to the user U-4 and the user image C-5 corresponding to the user U-5 indicate that conversation is not possible except for urgent matters. For example, when conversation is not possible even for urgent matters (when the evaluation value is 1), the user image Cj is displayed with a displeased facial expression. In FIG. 8, the user image C-2 corresponding to the user U-2 and the user image C-6 corresponding to the user U-6 indicate that conversation is not possible even for urgent matters.

[0071] The user image Cj may be displayed in a different color depending on the state of the user Uj. For example, the user image Cj may be color-coded like a traffic light, with blue when the user Uj is available for conversation, yellow when the user Uj is unable to talk except for urgent matters, and red when the user Uj is unable to talk even for urgent matters.

[0072] In this way, by displaying the status information using the facial expression or display color of the icon, the user can intuitively understand whether each user Uj is available for conversation. For example, when user U-1 sees the other user information S in Fig. 8, he / she understands that user U-3 is available for conversation and can talk to user U-3. Also, for example, when user U-1 has something to say to user U-6, since user U-6 is currently unavailable for conversation, he / she can consider talking to user U-6 at a later time or contacting user U-3 by e-mail, etc.

[0073] In addition, when the status of user U-6, who is currently unable to talk, changes and becomes able to talk, other users U may be notified. Specifically, for example, the providing unit 212 displays a notification reservation icon near the image C-6 of the user who is unable to talk. For example, when user U-1 wants to talk with user U-6, he taps the notification reservation icon. When the status of user U-6 changes and becomes able to talk, the providing unit 212 notifies the user terminal 10-1 of user U-1 that user U-6 is now able to talk. By providing such a notification, user U-1 does not need to worry about other user information, and the efficiency of work can be improved.

[0074] In addition, the design of the user image Cj is not limited to that shown in FIG. 8. For example, the user image Cj may be a blurred icon. More specifically, when the user Uj is able to talk, the providing unit 212 displays a blurred icon obtained by blurring an image of the user Uj as the user image Cj. The image is taken once every few minutes, and the blurred icon is updated each time. In addition, when the user Uj is unable to talk, the providing unit 212 does not display the blurred icon, but displays a display that visually indicates that the user Uj is unable to talk as the user image Cj. By using the blurred icon as the user image Cj, the user Uj's state can be grasped more specifically, and the psychological burden when talking to the user Uj is reduced.

[0075] A-4. Operation of the processing device 203 9 is a flowchart showing the operation of the processing device 108 of the user terminal 10-1. The processing device 108 functions as a detection unit 111 and detects the current behavior of the user U-1 (step S101). The processing device 108 functions as a determination unit 112 and determines the group to which the current behavior belongs and determines a first evaluation value (step S102). The current behavior detected in step S101 and the group to which the current behavior belongs determined in step S102 are stored in the storage device 107 in association with the current time. The stored current behavior and the group to which the current behavior belongs are hereafter treated as past behavior and the group to which the past behavior belongs.

[0076] The processing device 108 functions as the correction unit 113 and corrects the first evaluation value based on the past behavior to determine the second evaluation value (step S103). The processing device 108 reads out the past behavior and the group to which the past behavior belongs from the storage device 107 and uses them in the process of step S103. If no correction is necessary, the correction unit 113 sets the first evaluation value as the second evaluation value as it is.

[0077] The processing device 108 functions as the estimation unit 114 and judges whether the second evaluation value is equal to or greater than a threshold value (step S104). If the second evaluation value is equal to or greater than the threshold value (step S104: YES), the processing device 108 estimates that the user U-1 is able to converse (step S105). If the second evaluation value is less than the threshold value (step S104: NO), the processing device 108 estimates that the user U-1 is unable to converse (step S106). The processing device 108 may estimate a more detailed state of the user U-1 (unable to converse except for urgent matters, or unable to converse even for urgent matters) based on the second evaluation value.

[0078] The processor 108 functions as the status information transmitting unit 115, transmits the status information to the server 20 (step S107), and ends the processing of this flowchart.

[0079] A-5. Summary of embodiments As described above, the user terminal 10-1 according to the embodiment estimates the state of user U-1 at the current time based on a second evaluation value obtained by correcting a first evaluation value for the current behavior of user U-1 based on the past behavior of user U-1. In other words, the user terminal 10-1 estimates the state of user U-1 based on not only the current behavior of user U-1 but also the flow of behavior from the past. Therefore, the state of user U-1 is estimated with higher accuracy.

[0080] Furthermore, to estimate the state of user U-1, it is necessary to detect the behavior of user U-1 using images captured of user U-1, but some users U are reluctant to send this data to the server 20. For example, if there is a cyber attack on the server 20, there is a possibility that privacy risks will occur, such as information that can identify individuals (videos or images, etc.) being leaked. In the system 1, the user terminal 10-1 estimates the state of user U-1, and only the result is sent to the server 20 as state information. Therefore, there is no need to send information that can identify the behavior of user U-1 to the server 20, and the privacy of user U-1 is protected.

[0081] Furthermore, when multiple actions belonging to different groups are performed during a given period, the user terminal 10-1 corrects the content of the past actions based on the duration of each of the multiple actions. This allows the evaluation value to reflect the tendency of the user U-1's actions over a longer period.

[0082] Furthermore, the user terminal 10-1 corrects the first evaluation value and determines the second evaluation value based on the relationship between the type of the current action and the type of the past action, and can change the second evaluation value based on the combination of the current action and the past action.

[0083] In addition, the user terminal 10-1 estimates whether or not it is possible to talk to the user U-1 as the state of the user U-1. For example, even if another user U-2 is in a location far away from the user U-1, the other user U-2 can refer to the estimation result to know whether or not it is possible to talk to the user U-1. This can reduce the psychological burden of conversation between users U who are in different locations.

[0084] Furthermore, the user terminal 10-1 groups actions into actions of sending information, actions of acquiring information, and other actions. Therefore, the state of the user U who is performing a task using an information processing device can be estimated with high accuracy.

[0085] B: Variation The following are modified aspects of the above-described embodiment. Two or more aspects selected from the following modified aspects may be combined as appropriate within the scope of not being mutually contradictory.

[0086] B1: First modified example In the above embodiment, the availability of conversation with the user U-1 is estimated as the status of the user U-1. However, the status of the user U-1 is not limited to this, and may be, for example, work efficiency or learning efficiency.

[0087] Specifically, the "unable to talk" state in the embodiment can be rephrased as a state in which user U-1 is highly concentrated. Also, the "able to talk" state can be rephrased as a state in which user U-1 is relaxed. Therefore, by calculating the proportion of the "unable to talk" state and the "able to talk" state in the work time or study time, the work efficiency or study efficiency of user U-1 can be estimated. According to the first modified example, the work efficiency or study efficiency of user U-1 can be accurately estimated.

[0088] B2: Second variant In the above-described embodiment, the state of the user U-1 is estimated by the user terminal 10-1. However, the present invention is not limited to this, and the state of the user U-1 may be estimated by the server 20, for example. In this case, the user terminal 10-1 transmits to the server 20 an image of the user U-1 captured by the imaging device 104, a usage state of a keyboard and a mouse which are examples of the input device 102, a detection value of a sensor which is an example of the input device 102, a communication state of the communication device 103, and a log of the user terminal 10-1 (processing device 108). The server 20 includes a configuration corresponding to the detection unit 111, the determination unit 112, the correction unit 113, and the estimation unit 114, and estimates the state of the user U-1 based on information received from the user terminal 10-1. According to the second modification, the processing load of the user terminal 10-1 can be reduced.

[0089] B3: Third variant In the above embodiment, the users U-1 to Uk are assumed to be colleagues at work. The relationship between the users U-1 to Uk is not limited to this, and may be any group of people who agree to share each other's status information. For example, the users U-1 to Uk may be friends, people with a common hobby, or participants in a project that spans multiple companies.

[0090] B4: Fourth variant For example, when user U-1 talks to user U-3 using a messenger application or the like based on other user information (status information) as shown in Fig. 8, a sudden event (such as a call from a client) may occur on the user U-3 side, making it impossible to have a conversation. In such a case, it may be possible to send a message from user U-3 to user U-1 instructing him or her to follow the other person.

[0091] Specifically, for example, after user U-3 disconnects the messenger application, a message dialog for inputting a message to user U-1 who has rejected the call is displayed on the user terminal 10-3. The message dialog may be in a format in which user U-3 freely inputs text, or may be in a format in which user U-3 selects a fixed phrase. User U-3 briefly inputs the reason why he or she cannot talk to user U-1, for example, "Sorry, there is an outside line." After inputting the message, when user U-3 instructs to send the message (for example, when pressing the send button), the message is sent to the user terminal 10-1. The user terminal 10-1 displays the content of the message from user U-3 in a notification banner or the like.

[0092] According to the fourth modification, when the other user information (status information) is different from the actual situation and conversation is not possible, a message to follow the other party can be easily sent. This reduces the psychological burden of conversation between users U in different locations.

[0093] C:Other (1) In the above-described embodiment, ROM and RAM, etc. are given as examples of storage devices 107 and 202, but storage devices 107 and 202 may also be a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disc), a smart card, a flash memory device (e.g., a card, a stick, a key drive), a CD-ROM (Compact Disc-ROM), a register, a removable disk, a hard disk, a floppy (registered trademark) disk, a magnetic strip, a database, a server, or any other suitable storage medium.

[0094] (2) In the above-described embodiments, the described information, signals, etc. may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0095] (3) In the above-described embodiment, the input / output information, etc. may be stored in a specific location (e.g., memory) or may be managed using a management table. The input / output information, etc. may be overwritten, updated, or added. The output information, etc. may be deleted. The input information, etc. may be transmitted to another device.

[0096] (4) In the above-described embodiments, the determination may be made based on a value represented by 1 bit (0 or 1), a Boolean (true or false) value, or a numerical comparison (e.g., comparison with a predetermined value).

[0097] (5) The order of the process steps, sequences, flow charts, etc. illustrated in the above-described embodiments may be changed without causing any inconsistency. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0098] (6) Each function illustrated in Figures 2 and 3 is realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized by using one device that is physically or logically combined, or may be realized by using two or more devices that are physically or logically separated and directly or indirectly connected (for example, by using wires, wirelessly, etc.). The functional block may be realized by combining the one device or the multiple devices with software.

[0099] (7) The programs exemplified in the above-described embodiments should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., regardless of whether they are called software, firmware, middleware, microcode, hardware description language, or by other names.

[0100] Additionally, software, instructions, information, etc. may be transmitted or received over a transmission medium. For example, if the software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave, etc.), then these wired and / or wireless technologies are included within the definition of transmission media.

[0101] (8) In each of the above embodiments, the terms “system” and “network” are used interchangeably.

[0102] (9) The information, parameters, etc. described in this disclosure may be expressed using absolute values, relative values ​​from a predetermined value, or corresponding other information.

[0103] (10) In the above-described embodiments, the portable device may be a mobile station (MS). A mobile station may be referred to by those skilled in the art as a subscriber station, a mobile unit, a subscriber unit, a wireless unit, a remote unit, a mobile device, a wireless device, a wireless communication device, a remote device, a mobile subscriber station, an access terminal, a mobile terminal, a wireless terminal, a remote terminal, a handset, a user agent, a mobile client, a client, or some other suitable term. In this disclosure, the terms "mobile station", "user terminal", "user equipment (UE)", "terminal", etc. may be used interchangeably.

[0104] (11) In the above-mentioned embodiments, the terms "connected" and "coupled" or any variation thereof refer to any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, "connected" may be read as "accessed." As used in this disclosure, two elements may be considered to be "connected" or "coupled" to each other using at least one of one or more wires, cables, and printed electrical connections, as well as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and light (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

[0105] (12) In the above embodiments, the phrase "based on" does not mean "based only on," unless otherwise specified. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0106] (13) As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, search, inquiry (e.g., searching in a table, database, or other data structure), ascertaining, and the like. Also, "determining" and "determining" may include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in a memory), and the like. In addition, "judgment" and "decision" can include considering resolving, selecting, choosing, establishing, comparing, etc., to be a "judgment" or "decision." In other words, "judgment" and "decision" can include considering some action to be a "judgment" or "decision." In addition, "judgment (decision)" can be interpreted as "assuming," "expecting," "considering," etc.

[0107] (14) In the above embodiments, when "include," "including," and variations thereof are used, these terms are intended to be inclusive, similar to the term "comprising." Furthermore, the term "or" used in this disclosure is not intended to be an exclusive or.

[0108] (15) In this disclosure, where articles have been added by translation, such as a, an, and the in English, the disclosure may include that the nouns following these articles are in the plural.

[0109] (16) In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combined" may also be interpreted in the same way as "different."

[0110] (17) Each aspect / embodiment described in this disclosure may be used alone, in combination, or switched according to execution. In addition, notification of specific information (e.g., notification that "X is the case") is not limited to being done explicitly, but may be done implicitly (e.g., not notifying the specific information). A combination of a plurality of aspects selected from the above may be used. [Explanation of symbols]

[0111] 1...system, 10 (10-1 to 10-k)...user terminal, 20...server, 111...detection unit, 112...determination unit, 113...correction unit, 114...estimation unit, 115...status information transmission unit, 116...other user information acquisition unit, 211...status information acquisition unit, 212...providing unit, N...network, U (U-1 to Uk)...user.

Claims

1. A detection unit that detects a user's behavior; a determination unit that determines a first evaluation value for a current behavior of the user by determining to which of a plurality of groups hierarchized according to evaluation values ​​indicating a state of the user the current behavior of the user at a current time belongs; a correction unit that determines a second evaluation value by correcting the first evaluation value based on a past behavior of the user during a predetermined period of time in the past; an estimation unit that estimates a state of the user at the current time based on the second evaluation value; Equipped with A user state estimation device.

2. the plurality of groups includes a first group and a second group, the correction unit determines the second evaluation value by making the first evaluation value closer to an evaluation value of the second group when the past behavior belongs to the second group and the current behavior belongs to the first group. The user state estimation device according to claim 1 .

3. the correction unit corrects the content of the past behavior based on a duration of each of the multiple behaviors when multiple behaviors belonging to different groups are performed during the predetermined period. The user state estimation device according to claim 1 .

4. each of the plurality of groups includes a plurality of types of behavior; the correction unit corrects the first evaluation value based on a relationship between the type of the current action and the type of the past action. The user state estimation device according to claim 1 .

5. The estimation unit estimates, as the state of the user, whether or not a conversation with the user is possible. The user state estimation device according to claim 1 .

6. the evaluation value is a tolerance for conversation with the user, The plurality of groups are hierarchically divided into group A, group B, and group C in ascending order of the evaluation value, The group A includes the behavior of transmitting information, Group B includes actions to obtain information; The group C includes behaviors other than the behavior of transmitting the information and the behavior of acquiring the information, The user state estimation device according to claim 5 .

7. A user state sharing system including a server, a first user terminal, and a second user terminal, The first user terminal, A detection unit that detects a behavior of a first user; a determination unit that determines a first evaluation value for a current behavior of the first user by determining to which of a plurality of groups hierarchized according to evaluation values ​​indicating a state of the first user the current behavior of the first user at a current time belongs; a correction unit that determines a second evaluation value by correcting the first evaluation value based on a past behavior of the first user in a predetermined period of time in the past; an estimation unit that estimates a state of the first user at the current time based on the second evaluation value; a status information transmission unit that transmits status information indicating a status of the first user to the server, The server, a providing unit for providing the state information to the second user terminal; User state sharing system.

Citation Information

Patent Citations

  • Display control apparatus, display control method, and display control program

    JP2022095256A