Virtual space video calling system and method

Abstract

To improve user-friendliness for a user.SOLUTION: A virtual space video call system 1 includes a virtual space control unit 21, a video call control unit 22 which controls a video call, and a call candidate management unit 25 which manages candidates of video call. The call candidate management unit includes: a user selection unit 250 which displays user information, as candidates of a video call, to be selected; a candidate user status confirmation unit 251 which displays a status of the candidate users; and a moving unit 252 which moves an avatar of a user into a predetermined range of a candidate avatar. The virtual space control unit determines whether the avatars in the virtual space have made contact with each other. The video call control unit causes the avatars of which contacts are determined by the virtual space control unit to start a video call between client terminals 3 corresponding to the avatars.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a virtual space video calling system and method. [Background technology]

[0002] There is known a technology in which a plurality of users gather in a virtual space provided on a server on a network and interact with each other through chats, games, and the like (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 5969476 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in recent years, with the spread of telework, where people work from home, there has been a lack of opportunities for communication, resulting in problems with maintaining a sense of camaraderie in the workplace and sharing information.

[0005] One idea is to gather employee avatars in a virtual office and hold video calls, but with a typical video calling system, you select the person you want to call from a list and press the call start button, regardless of the location of the avatar in the virtual office. Therefore, even if video calling is introduced in a virtual office, it is not much different from making a video call on a smartphone, and it lacks the ease and convenience of going up to the person you want to talk to and talking to them, as is often done in real offices.

[0006] Incidentally, since the amount of data required for voice calls is small, it is also conceivable that other avatars within a circular calling range set for an avatar may be automatically invited to a voice channel to start a voice call.

[0007] However, video calls require a larger amount of data than voice calls. Therefore, if a method were to automatically start a video call with another avatar within the calling range, it would consume a large amount of computer resources and communication bandwidth. On the other hand, it would be possible to provide a button on the screen for starting a video call and start a video call when the button is pressed, but in that case, a special operation would be required to start the call, which is different from starting a conversation in an actual office, making it less user-friendly and not promoting casual communication.

[0008] SUMMARY OF THE PRESENT EMBODIMENT An object of the present invention is to provide a virtual space video calling system and method that can improve usability for users. [Means for solving the problem]

[0009] A virtual space video calling system according to one aspect of the present invention is a virtual space video calling system in which multiple users make video calls via a virtual space, and includes a virtual space control unit that controls the virtual space provided to client terminals used by the multiple users, a video call control unit that controls video calls in the virtual space, and a call candidate management unit that manages specific users among the multiple users as candidates for the video call, and the call candidate management unit includes a user selection unit that displays information about the specific user and allows a candidate user to be the other party of the video call from among the specific users, a candidate user status confirmation unit that displays information indicating the status of the candidate users, and a movement unit that moves the avatar of the user wishing to make a video call within a specific range of the avatar of the candidate user selected by the candidate user selection unit.

[0010] When the video call control unit receives a specified instruction, it can start a video call between a client terminal used by the user who desires to make the video call and a client terminal used by the selected candidate user.

[0011] When the virtual space control unit determines that avatars in the virtual space have come into contact with each other, it sends the specified instruction to the video call control unit, and the video call control unit can start a video call between client terminals used by users corresponding to each avatar for the avatars determined by the virtual space control unit to have come into contact.

[0012] The information on the specified user may display a user status indicating the status of the specified user, and the candidate user status confirmation unit may display an avatar status indicating the surrounding status of an avatar of the candidate user.

[0013] When the avatar of a user wishing to make a video call moves within a specified range, the virtual space control unit can display a specified effect indicating that the avatar of the user wishing to make a video call is near the avatar of the candidate user.

[0014] The virtual space has a plurality of virtual floors, and the virtual floor on which the avatar wishing to make a video call is located may be different from the virtual floor on which the avatars of the candidate users are located.

[0015] The call candidate management unit can also display the number of viewers, which is the number of users currently viewing the avatar of the user who wishes to make a video call.

[0016] The call candidate management unit can also display the number of users viewing the avatar of a user who wishes to make a video call according to the relationship between the user viewing the avatar and the user who wishes to make a video call.

[0017] The user situation may be created based on information acquired from a management system that manages the user's work situation and information acquired from a client terminal used by the user.

[0018] The video call control unit can start a video call by displaying, within each avatar that comes into contact in the virtual space, an image obtained from a client terminal used by a user corresponding to each avatar. [Brief description of the drawings]

[0019] [Figure 1] 1 is an overall schematic diagram of a virtual space video calling system according to an embodiment of the present invention; [Diagram 2] FIG. 1 is an explanatory diagram of the configuration of a virtual office. [Diagram 3] FIG. 1 is an explanatory diagram showing how a user who wishes to make a video call selects candidate users for the video call; [Figure 4] FIG. 13 is an explanatory diagram showing how a video call is started when the avatar of a user desiring to make a video call moves close to the avatar of a candidate user and the two avatars come into contact. [Diagram 5] FIG. 1 is a functional diagram of a virtual space video calling system. [Figure 6] 10 is a flowchart showing a process of collecting information from a client terminal. [Figure 7] 13 is a flowchart of the entire process. [Figure 8] 13 is a flowchart of a video call process. [Figure 9] Example of follow-up information. [Figure 10] An example of information that indicates the user's status. [Figure 11] An example of an interest filter. [Figure 12] 10 is a flowchart showing a virtual office display process according to the second embodiment. [Figure 13] FIG. 13 is an explanatory diagram showing how information about a user viewing, from another floor, the avatar of a user who wishes to make a video call is notified. [Figure 14] 13 is a flowchart showing a process of analyzing a user's situation based on information acquired from a client terminal and information acquired from another system according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] In this embodiment, a virtual space video calling system is provided that enables a user wishing to make a video call to easily make a video call with another user, just like having a conversation in a real office, by simply intuitively moving his or her avatar toward the avatar of the person he or she wants to talk to and bringing it into contact with the avatar.

[0021] Furthermore, in this embodiment, even if the virtual space is large or made up of multiple floors, it is possible to easily check the appearance of the person with whom you wish to make a video call before moving, and it is also possible to quickly move close to that person. EXAMPLES

[0022] A first embodiment will be described with reference to Figures 1 to 11. Figure 1 shows an overview of a virtual space video calling system 1 according to this embodiment. The virtual space video calling system 1 includes, for example, a virtual office providing server 2 and a plurality of client terminals 3.

[0023] The virtual office providing server 2 provides a virtual office 4 as a "virtual space" to each client terminal 3. The virtual office providing server 2 includes a virtual office control unit 21 serving as a "virtual space control unit" that controls the virtual office 4, a video call control unit 22 that controls video calls made in the virtual office, an information gathering unit 23, an information providing unit 24, and a call candidate management unit 25. The virtual office providing server 2 is also connected for two-way communication with another system 5 serving as a "management system".

[0024] The virtual office control unit 21, which serves as a "virtual space control unit," provides a virtual office 4, which is an example of a "virtual space," to each client terminal 3. A user can use the client terminal 3 to operate an avatar corresponding to the user in the virtual office 4.

[0025] The video call control unit 22 controls a video call that is performed between users through avatars in the virtual office 4. As described below, in this embodiment, when an instruction is received that the avatars have come into contact with each other in the virtual office 4, the video call is started.

[0026] In the following, a user who wishes to make a video call may be referred to as the "own user," an avatar corresponding to the own user may be referred to as the "own avatar," avatars corresponding to candidate users for a video call may be referred to as "candidate avatars," and users other than the own user may be referred to as "other users."

[0027] The information collecting unit 23 collects information from each client terminal 3. For example, the information collecting unit 23 acquires the type of operating system and application running on the client terminal 3, remaining memory capacity, CPU usage, operations of an input device such as a keyboard, a face image of the user, and the like from the client terminal 3. The information collecting unit 23 may acquire vital information such as body temperature, pulse rate, and blood pressure from the client terminal 3.

[0028] The information providing unit 24 distributes all or part of the information collected from each client terminal 3 to each client terminal 3.

[0029] The call candidate management unit 25 manages predetermined users who are candidates for a video call. The call candidate management unit 25 includes a follow list 250 as a "user selection unit", a confirmation button 251 as a "candidate user status confirmation unit", and a move button 252 as a "move unit".

[0030] As described later, the follow list 250 displays predetermined users who are candidates for a video call and the status of the predetermined users, and the user can select one desired predetermined user from the follow list 250.

[0031] The user can display the status of the avatar corresponding to the selected user by operating the confirmation button 251. The user can move his / her avatar close to the avatar of the selected user by operating the movement button 252. The user can then start a video call by moving his / her avatar towards the avatar of the selected user who is nearby and bringing it into contact with the avatar of the selected user.

[0032] The other systems 5 include, for example, project management, task management, workflow management, groupware, schedule management, file sharing, email management, attendance management, etc. Although only one other system 5 is shown in Fig. 1, the virtual office providing server 2 can be connected to a plurality of different other systems 5 for two-way communication.

[0033] In the following description, the virtual space video calling system 1 may be abbreviated as the video calling system 1, and the virtual office providing server 2 may be abbreviated as the server 2.

[0034] A user accesses the server 2 using a client terminal 3 and logs in to the virtual office service, thereby making his / her avatar GP1 appear in the virtual office 4. As will be described later, the client terminal 3 has a camera as an image acquisition unit in its user interface unit 33.

[0035] The virtual office 4 is a virtual space used by an organization to which the user belongs. The layout of the virtual office 4 can be similar to that of a real office, or can be different from that of a real office. Organizations to which the user belongs include, for example, companies, government agencies, local public organizations, and non-profit organizations (NPOs).

[0036] In the virtual office 4, for example, furniture and fixtures such as a desk GP2 and a chair GP3, a desk-top nameplate GP4, a conference room GP5, and a concentration room GP6 (see FIG. 2) are drawn. While the avatar GP1 is away from a predetermined seat, a ghost avatar GP12 is displayed at that seat. The ghost avatar GP12 indicates that the user is logged in but is not at that seat. For example, in FIG. 1, the avatar of the user U3 is not at his / her seat because he / she has gone to another floor. At this time, the ghost avatar GP12 is displayed instead of the avatar GP1 (U3). When the avatar of another user comes into contact with the ghost avatar GP12, the avatar of the other user is moved to the location of the avatar GP1 (U3) corresponding to the ghost avatar GP12. Although not shown in the figure, the avatar used by the user U3 is denoted by the symbol GP1 (U3) for the explanation. The same applies to the following explanation.

[0037] For example, a case will be described where the virtual office 4 has multiple floors, such as a first floor 4 (F1) and a second floor 4 (F2). When an avatar GP1 (U3) on the first floor 4 (F1) moves to the second floor 4 (F2), if another avatar GP1 on the first floor 4 (F1) comes into contact with a ghost avatar GP12 (U3), that avatar GP1 moves close to the avatar GP1 (U3) on the second floor 4 (F2).

[0038] An information display section GP11 showing information about the user corresponding to the avatar is displayed in association with the avatar GP1. The information display section GP11 includes, for example, the name of the user operating the avatar GP1. In addition to the name, the information display section GP11 can also display the user's status, the name of the department to which the user belongs, a numerical value showing the activity of the user's actions, information showing the user's status, and the like, as "user status."

[0039] The user's status may include, for example, the user's own situation, such as on the phone, in a meeting, busy, off work, vacation, etc. The user's status may further include, for example, biological information and mental information, such as physical condition, sleep time, body temperature, heart rate, fatigue level, happiness level, etc.

[0040] As will be described later with reference to FIG. 3, information indicating the user's state (information indicating the user's situation) is provided to other users using a plurality of information display sections GP11, GP31, and GP32.

[0041] FIG. 2 shows an example of a virtual office 4. The virtual office 4 is configured as a virtual two-dimensional space or a virtual three-dimensional space. The virtual office 4 may be configured as a so-called virtual 2.5-dimensional space. FIG. 2 shows a relatively small-scale virtual office 4 on one floor equipped with four desks GP2 and 16 chairs GP3, but the virtual office 4 is not limited to this and may be a large-scale virtual office 4 equipped with multiple floors, each of which can accommodate tens to hundreds of people.

[0042] The conference room GP5 is an area where users hold a conference through avatars. Who is in the conference room GP5 can be visually confirmed from areas other than the conference room GP5. When the avatar GP1 enters the conference room GP5, the avatar automatically becomes a participant in the video call. For example, when the avatar GP1 (U12) enters the conference room GP5 where the avatar GP1 (U8) is present, a video call between the avatars starts. In this embodiment, a video call is started when the avatars GP1 come into contact with each other, but a video call can be started in the conference room GP5 even if the avatars do not come into contact with each other. This is because the conference room GP5 is an area for a conference, and it is clear that the avatar wishes to make a video call at the time of entering the conference room. However, a video call may also be started in the conference room GP5 when the avatars come into contact with each other.

[0043] The concentration room GP6 is an area used by users who want to concentrate on work or thinking. Only one avatar can enter the concentration room GP6. Other avatars cannot contact the avatar in the concentration room GP6, so they cannot make video calls with the avatar who has entered the concentration room GP6. For example, it is possible to contact the avatar who has entered the concentration room GP6 by displaying a message on the desk nameplate GP4.

[0044] The avatar GP1 may have a common design for all users, or may be selectable for each part, such as hairstyle or accessories. The avatar GP1 does not need to be designed to resemble a human, and may be an animal, plant, robot, machine, figure, or the like. There may be multiple types of avatars, such as a normal avatar GP1, a ghost avatar GP12, and a busy avatar GP1N. The busy avatar GP1N is an avatar for informing people around that the user is busy and will not accept video calls from other users. Therefore, even if another user's avatar GP1 comes into contact with the busy avatar GP1N, a video call will not be started.

[0045] 3 and 4, a case where a video call is made in a virtual office 4 will be described. A follow list 250 is displayed in the virtual office 4. The follow list 250 is information in which predetermined users who are candidates for a video call are registered, and is provided in a list format. The display of the follow list 250 may be turned on and off by a user's operation, or may be constantly displayed.

[0046] In the follow list 250, the status of users registered as "predetermined users" is displayed in the information display section GP11. Therefore, the user can quickly grasp the status of registered users (hereinafter also referred to as registered users) just by glancing at the follow list 250. The user can select a partner for a video call from among the registered users who are in a situation where a video call is possible.

[0047] The display order in the follow list 250 may be changed according to the status of the registered user. For example, among the registered users who are online, those who are available for video calls may be displayed at the top of the follow list 250. A registered user who is available for video calls is a registered user who is not in a specific situation in which it is determined that a video call is not possible, such as being away from his / her desk or on the phone. Alternatively, a registered user who is only displayed as being online and has no other status displayed may be set as a registered user who is available for video calls.

[0048] Although there is no limit to the number of users that can be registered in the follow list 250, an upper limit of several to several tens of people may be set in order to save network and computer resources. An upper limit on the number of users that can be registered in the follow list 250 may be set according to the attributes of the user, such as the job title or department to which the user belongs.

[0049] Although users on the same floor may be registered in the follow list 250, it is more convenient to register users on different floors. A user in the virtual office 4 (F1) on the first floor can easily check the status of the other party or start a video call by registering a user in the virtual office 4 (F2) on another floor (for example, the second floor).

[0050] Even in the case of a large virtual office on one floor, by registering other users on the same floor in the follow list 250, a user who wishes to make a video call can check the appearance of the avatar of the user with whom he or she wants to make a video call without having to scroll the screen multiple times, and can easily start a video call without having to move his or her own avatar a long distance.

[0051] 3 shows a case where user U3, whose avatar is in virtual office 4 (F1), wishes to have a video call with user U28, who is registered in follow list 250. In the following description, user U3 is the user who wishes to have a video call, and user U28 is a candidate user for the video call. It is assumed that user U28 is in a virtual office 4 (F2) different from the virtual office 4 (F1) in which user U3's avatar is located.

[0052] As described above, the situation (user status) of the user U28 is displayed in the follow list 250. Therefore, by checking the follow list 250, the user U3 can easily grasp the general situation of the user U28.

[0053] However, there are cases where the actual situation of the call candidate user U28 cannot be grasped only by the information obtained from the information display unit GP11. For example, when the avatars of other users are gathering near the avatar of the user U28 and it looks like a video call is about to start, if the avatar of the user U28 is in the concentration room GP6, the status of the user U28 is "in a meeting", but the avatar of the user U28 is alone in the conference room GP5, the situation of the user displayed in the follow list 250 may not necessarily match the situation of the avatar corresponding to the user. Therefore, unless the situation of the avatar of the call candidate user is also confirmed, it may not be possible to appropriately determine whether or not a video call is possible.

[0054] In response to this, it is also possible to classify the user's status in more detail, such as "Staying alone in a conference room" or "Likely to start chatting over video call", and display the classified status on the information display unit GP11. In this case, it is possible to determine whether a user registered in the follow list 250 is available for video call just by looking at the follow list 250. However, since it is necessary to grasp the status of all users under the management of the call system 1 and manage their status in detail, this consumes a large amount of computer and network resources.

[0055] Therefore, in this embodiment, as shown in the upper part of Fig. 3, a "Confirm" button 251 is provided in the follow list 250. The user U3 operates the confirm button 251 in a state where the user U3 selects a party with whom he / she wishes to have a video call (here, the user U28).

[0056] Then, as shown in the lower part of FIG. 3, a surrounding situation window W1 showing the surrounding situation of the avatar corresponding to the video call candidate U28 is displayed on the screen of the client terminal 3 used by the user U3.

[0057] The window W1 may be displayed over the virtual office 4 (F1) where the avatar of the user U3 is located, or the image of the virtual office 4 (F1) may be switched to the surrounding situation window W1. The surrounding situation window W1 is a display area that displays the surrounding area of ​​the avatar corresponding to the candidate user U28 for the video call. The surrounding area of ​​the avatar corresponding to the candidate user U28 is an area within a predetermined range from the avatar. The shape of the surrounding area may be any shape, such as a rectangle, a circle, or an ellipse. The surrounding situation window W1 can display an enlarged view of the avatar of the candidate user for the video call and its surroundings. Furthermore, at least a part of the environment of the virtual office 4 (F2), such as sound, light, messages, and vibrations, that are expressed around the avatar of the candidate user, may be reproduced in the surrounding situation window W1. For example, the situation in which the candidate user U28 is placed can be determined from the music, voice, text messages, and color of the lighting in the room that are playing in the virtual office 4 (F2) where the candidate user U28 is located.

[0058] Furthermore, information display areas GP11 (omitted in FIG. 3), GP31, and GP32 showing the user's situation are displayed around the avatar of the candidate user U28. Information showing the user's situation is displayed in these information display areas.

[0059] The surrounding situation window W1 also displays a move possibility window W2 for moving the avatar of user U3 closer to the avatar of candidate user U28. User U3, who wishes to make a video call, can move the avatar of user U3 closer to the avatar of user U28, who is a candidate for the video call, by operating the "Yes" button B1. If user U3 operates the "No" button B2, the avatar will not move. When user U3 operates the "No" button B2, the surrounding situation window W1 may be dismissed. When user U3 operates the "Yes" button B1, the avatar moves to the diagram shown in the upper part of FIG. 4.

[0060] In addition, user U3 can also make his / her own avatar appear near user U28's avatar by operating movement button 252 displayed on follow list 250 without checking the surrounding situation of user U28's avatar.

[0061] The display periphery of the window W1 may or may not match the range in which the avatar of the user U3 appears when the user U3 who wishes to make a video call operates the movement button 252.

[0062] As shown in the upper part of Fig. 4, the avatar of user U3 who wishes to make a video call moves close to the avatar of user U28 who has been selected as the call partner. The avatar of user U3 appears at a location a predetermined distance ΔL away from the avatar of user U28. A predetermined effect GP13 is displayed on the avatar of user U3 to notify the surroundings of the presence of the avatar of user U3.

[0063] The predetermined effect may be called a "notification effect", a "video call request effect", etc. Examples of the predetermined effect include sound, a voice message, a color change, a change in the size of the avatar, vibration (in the case where a vibration device is connected to the client terminal 3), and a combination of these. The effect is not limited to these effects, and any effect that can directly or indirectly notify the user U28 of the approach of the user U3 may be adopted.

[0064] As shown in the lower part of Fig. 4, user U3 who wishes to make a video call moves his / her avatar towards the avatar of user U28, bringing the two avatars into contact. This opens a video call channel and starts the video call. Hereinafter, the user's code may be added to the avatar code GP1, such as avatar GP1(U3), to distinguish which user's avatar it belongs to.

[0065] When the avatars come into contact with each other, the video call control unit 22 starts preparing for the video call. Until the video call is ready, the virtual office control unit 21 can display an effect (not shown) indicating that the video call is being prepared on each of the avatars GP1 (U3) and GP1 (U28) that are the targets of the video call. The virtual office control unit 21 notifies not only the targets of the video call U3 and U28 but also other users in the vicinity that the video call has started by, for example, changing the color of the avatar, making it blink, enlarging or reducing the size, or displaying an animation.

[0066] When a video call is started, the size of each of the avatars GP1 (U3) and GP1 (U28) participating in the video call is enlarged by a predetermined amount. The video VD of the user is displayed in the enlarged avatar. That is, the video VD including the face image of the user U3 is displayed in the enlarged avatar GP1 (U3). Similarly, the video VD of the user U10 is displayed in the enlarged avatar GP1 (U28).

[0067] Whether or not the user's face image is displayed in the avatar GP1 depends on the orientation of the camera of the client terminal 3, work regulations, etc. If the user does not wish to show his or her face, the user can turn off the camera, point the camera in a direction away from the user, or display a still image or animation instead of the face image.

[0068] The avatar during a video call appears in normal size to other users who are not participating in the video call. The color, shape, or size of the avatar during a video call may be changed to clearly inform other users that a video call is in progress. A user who wishes to participate in the video call can join the video call by touching his or her avatar to either avatar GP1 (U3) or GP1 (U28) during the video call.

[0069] As described above, in this embodiment, a video call is started when avatars come into contact with each other, but the contact between avatars is not limited to the case where the outer edges of the avatars come into contact with each other, but also includes the case where the distance between the outer edges of the avatars becomes equal to or smaller than a predetermined distance. The avatars may partially overlap each other.

[0070] The functional configuration and hardware configuration of the video calling system 1 will be described with reference to Fig. 5. As described in Fig. 1, the video calling system 1 includes a virtual office providing server 2 and a plurality of client terminals 3. The server 2 is connected to each of the client terminals 3 and other systems 5 via a communication network CN so as to be capable of two-way communication.

[0071] The server 2 is a computer equipped with a processor 200 and a memory 201, and functions as the virtual office providing server 2 by executing a computer program stored in the memory 201.

[0072] The server 2 includes, for example, a virtual office control unit 21, a video call control unit 22, an information collecting unit 23, an information providing unit 24, a call candidate management unit 25, a database (DB in the figure) 26, and a communication unit 27. Although lines are omitted in the figure, the virtual office control unit 21, the video call control unit 22, the information collecting unit 23, and the information providing unit 24 are connected to the database 26.

[0073] The database 26 includes, for example, a table for managing authentication information of each user who uses the virtual office service, a table for managing each user's virtual office usage history, a table for managing the positions and display forms of various objects placed in the virtual office 4, and a table for managing user status information obtained from the information collection application 32 of each client terminal 3 (all not shown).

[0074] The communication unit 27 is a function for performing two-way communication with each client terminal 3 via the communication network CN.

[0075] The storage medium MM is a storage medium capable of retaining data for a relatively long period of time, such as a flash memory or a hard disk drive. By connecting the storage medium MM to the server 2, at least a portion of a computer program can be transferred between the storage medium MM and the memory 201.

[0076] The client terminal 3 is a computer used by a user. The client terminal 3 is configured as, for example, a laptop personal computer, a desktop personal computer, a tablet personal computer, a portable information terminal (including a so-called smartphone), a wearable terminal, or the like.

[0077] The client terminal 3 includes, for example, a virtual office application 31, an information collection application 32, a user interface (UI in the figure) unit 33, and a communication unit .

[0078] The virtual office application 31 is a computer program for operating the avatar GP1 in the virtual office 4. The user interface unit 33 is a function for the user and the client terminal 3 to input and output information via an information input / output device (not shown). Examples of the information input / output device include a keyboard, a pointing device, a microphone, a camera, a monitor display, a printer, and a speaker. The communication unit 34 has a function for two-way communication with the server 2 via the communication network CN.

[0079] The information collection application 32 sends information obtained from the user interface unit 33 and information obtained from an operating system not shown or other applications running on the client terminal 3 to the information collection unit 23 of the server 2, either as is or after processing.

[0080] For example, an image captured by a camera can tell whether or not a user is in front of the client terminal 3. For example, a microphone signal can tell whether or not the user is talking, and the noise surrounding the user. For example, by acquiring keyboard operations, it can be known that the user is inputting some information into the client terminal 3. For example, by acquiring operations of a mouse or a touch panel, it can be known that the user is operating some application such as a web browser. For example, if the client terminal 3 has a GPS (Global Positioning System), the user's current location, moving direction, moving speed, moving trajectory, etc. can be known. For example, if the client terminal 3 has a motion sensor, the user's actions can be known.

[0081] Furthermore, it is possible to know the web pages the user has visited by acquiring the access history of other applications running on the client terminal 3, for example, a web browser. For example, it is possible to know what software the user has been using by acquiring information on word processing software, spreadsheet software, design support software, etc., operated from the client terminal 3.

[0082] Furthermore, by acquiring user information from a time attendance management system, it is possible to know what time the user came to work and what time the user left work. For example, by acquiring information from a building entrance / exit management system, it is possible to know which building the user entered, what time, and what time the user left.

[0083] As described above, in this embodiment, the status of the candidate user for the video call is displayed in the follow list 250, and when the confirmation button 251 is operated, the status of the avatar used by the candidate user is displayed in the window W1. The avatar status is information indicating the status of the avatar and its surroundings, and therefore can also be called "avatar surrounding information."

[0084] FIG. 6 is a flowchart showing a process in which the information collecting unit 23 of the server 2 collects information from the information collecting application 32 of each client terminal 3.

[0085] The information collecting application 32 acquires the status of the information input / output unit (keyboard, camera, microphone, etc.) from the user interface unit 33 (S11), and acquires the status of other applications (web browser, clerical processing application, software development application, etc.) running on the client terminal 3 (S12). Furthermore, the information collecting application 32 acquires the status of the operating system of the client terminal 3 (S13), and acquires the location information of the client terminal 3 (S14). The information collecting application 32 transmits the acquired information to the information collecting unit 23 of the server 2 as user status information (S15).

[0086] When the information collection unit 23 receives information from each client terminal 3 (S21), it stores the received information in the database 26 (S22) and analyzes it (S23). Here, analysis refers to processing information from the information collection application 32 to determine the user's situation. For example, it is processing to determine how busy the user is from the frequency of inputting characters into the keyboard. In the embodiment described later, the situation of the user is determined from workflow, to-do list, schedule data, etc. related to the user. This determination can be realized by utilizing so-called artificial intelligence such as deep learning and machine learning.

[0087] The information collecting unit 23 stores the analysis result in step S23 in the database 26 (S24). In step S21, the information collecting unit 23 can also acquire information from another system 5. For example, the information collecting unit 23 can specify the identification number of the user who operates the client terminal 3, inquire about the attendance status of the user to the attendance management system, and acquire the response.

[0088] 7 is a flowchart of the overall processing of the video calling system 1. When the virtual office control unit 21 is accessed from the client terminal 3 and acquires login information (information used for user authentication) from the user, it executes a login process to the virtual office service (S31). The virtual office control unit 21 executes a virtual office display process that places objects such as avatars and desks in the virtual office 4 (S32).

[0089] The information providing unit 24 refers to an interest information filter T2 (described later in FIG. 11) set for the logged-in user (S33), and displays information (user situation information) set as being of interest to the user about other users around the other avatars (S34). An example of displaying the situation of a user operating an avatar around an avatar has been described with reference to FIG. 3.

[0090] When the virtual office control unit 21 receives an operation from the client terminal 3 (S35), it controls the display of the virtual office 4 in response to the operation from the client terminal 3, such as changing the position or facial expression of the avatar (S36).

[0091] When the avatars come into contact with each other, the video call control unit 22 starts a video call process (S37). An example of the video call process will be described later. The virtual office control unit 21 executes this process until an instruction to stop the virtual office service is given (S38).

[0092] Fig. 8 is a flowchart showing details of the video call process S37 in Fig. 7. The video call control unit 22 determines whether a candidate user for a video call has been selected from among the users registered in the follow list 250 (S41).

[0093] When a candidate user for a video call is selected from among users registered in follow list 250 (S41: YES), video call control unit 22 determines whether confirmation button 251 has been operated (S42).

[0094] When the confirmation button 251 is operated to check the status of the avatar corresponding to the candidate user (S42: YES), the information providing unit 24 refers to the interest information filter T2 set for the user who wishes to make a video call, and displays information that is set as being of interest to the candidate user for the video call around the avatar of the candidate user (S44).

[0095] The virtual office control unit 21 accepts a movement operation by a user who wishes to make a video call (S45). This movement operation includes an operation for instantly moving one's avatar close to the candidate user's avatar (operation of the movement button 252) and an operation for moving one's avatar with a mouse or the like.

[0096] When the move button 252 is operated in step S45, the avatar of the user desiring to make a video call immediately moves close to the avatar of the candidate user for the video call, as described in the upper part of Fig. 4. The user desiring to make a video call can also move his / her avatar toward the avatar of the candidate user for the video call by scrolling the screen without using the move button 252.

[0097] The virtual office control unit 21 determines whether the avatar of the user who desires to make a video call has approached the avatar of the candidate user for the video call (S46). When the two avatars approach each other (S46: YES), the virtual office control unit 21 causes the avatar of the user who desires to make a video call to display an effect GP13 for notifying the approach (S47). The effect for notifying the approach may be realized by an element that is recognized not only visually but also by an element that is recognized by hearing or touch.

[0098] The virtual office control unit 21 determines whether the avatars have come into contact with each other (S48), and if the avatars have come into contact with each other (S48: YES), notifies the video call control unit 22 to that effect.

[0099] When the avatars come into contact with each other and an instruction to start a video call is given, the video call control unit 22 starts preparations for the video call, such as sharing video and audio, and starts the video call when the preparations are complete (S49).

[0100] In step S49, for example, the virtual office control unit 21 displays an effect indicating preparation for a call, enlarges the size of the avatar, sets a communication channel, etc. The video call control unit 22 displays the video VD captured by the camera of the client terminal 3 in the avatar, and starts a voice call (S49).

[0101] The virtual office control unit 21 determines whether the contact between the avatars has been resolved, i.e., whether the avatars during the video call have separated (S50). While the avatars remain in contact with each other (S50: NO), the video call is maintained (S49). When the avatars separate (S50: YES), the video call ends (S51).

[0102] Even if the contact between the avatars is resolved, the video call may be continued for a predetermined short period of time rather than being immediately terminated, for example, in case the user moves the avatar by mistake.

[0103] The user can also select a user who is not registered in the follow list 250 as a candidate user for a video call (S41: NO). The user moves his / her avatar to the location of the avatar of the candidate user for the video call (S45, S46: YES). When the user finds the avatar of the candidate user, the user brings his / her avatar into contact with the avatar of the candidate user (S48: YES) and starts the video call (S49).

[0104] If the avatar of the candidate user for the video call is near your avatar and you can check the status of the candidate user's avatar just by scrolling the screen a little, there is little need to use the follow list 250. However, if the virtual office is large or has multiple floors, you need to take the trouble to move your avatar to the location of the candidate user's avatar, and you cannot start the video call quickly. In this embodiment, you can register the candidate user in the follow list 250 and view (observe) the status of the candidate user's avatar without moving your avatar.

[0105] Fig. 9 shows an example of follow information T1. Although Fig. 9 shows one follow information T1, follow information T1 is created for each user. A follow list 250 is created and displayed based on the follow information T1.

[0106] The follow-up information T1 includes, for example, a user ID C11, a user name C12, an affiliated department C13, an affiliated project C14, a date added to the list C15, a confirmation date and time C16, a call date and time C17, and other information C18.

[0107] The user ID C11 is information for identifying each user managed by the video call system 1. The user name C12 is the name of a user registered as a candidate for a video call. The department C13 is the department to which the registered user belongs. The project C14 is the project to which the registered user belongs. The list addition date C15 is the date on which the user identified by the user ID was registered as a candidate for a video call. The confirmation date and time is the date and time on which the status of the avatar corresponding to the registered user was confirmed (the date and time on which the confirmation button 251 was operated). The call date and time C17 is the date and time on which a video call was made with the registered user. The other information C18 includes, for example, the cumulative time and the cumulative number of video calls. Note that at least some of the items of the follow information T1 shown in FIG. 9 may not be present.

[0108] A user can explicitly specify a candidate user for a video call to register the user in the follow list 250. For example, a user can register a user in the follow list 250 by selecting an avatar of the user to be registered and operating a registration button (not shown).

[0109] Alternatively, the user can detect the user to be registered by inputting the name or user ID of the user to be registered in a user search window (not shown), and register the detected user in the follow list 250. Alternatively, a user with whom the user has made a video call may be automatically registered in the follow list 250. Alternatively, users who belong to the same department as the user or users who belong to the same project as the user may be automatically registered in the follow list 250.

[0110] Fig. 10 is an explanatory diagram showing an example of user situation information, which shows examples of display types of user situation information, methods of acquiring the information, methods of expressing the user situation information, and examples of control thereof.

[0111] Display types of user status information include, for example, "1. Business status," "2. User status (status of the user in front of the client terminal)," "3. Online status," "4. Floor status," "5. Calendar status," and "6. Communication status."

[0112] Information regarding the "business status" is acquired, for example, from the operating status of a telephone application, the operating status of an online conference application, the operating status of an email application, the operating status of a software development application, the operating status of an office processing application, the on / off state of the screen of the client terminal 3, the on / off state of the power supply of the client terminal 3, the contents posted to a chat application or a social networking service (SNS), the position measurement function of the client terminal 3, etc. Depending on the contents of the business status, for example, a call mark, a meeting mark, an email mark, an under development mark, a document preparation mark, a client terminal sleeping mark, an unknown mark (when the client terminal is powered off), a moving mark, etc. are displayed around the avatar.

[0113] Information regarding the "user status" is acquired, for example, from the state of input to the keyboard, input sounds to the microphone, the presence or absence of an image of the user captured by the camera, the user's actions and expressions captured by the camera, user location information, etc. Depending on the content of the user status, for example, a sweating animation, a speech bubble animation, an away mark, a nodding mark, a current location mark (such as "head office", "factory", or "at home") is displayed around the avatar.

[0114] Information regarding the "online status" is obtained, for example, from the operation of a prepared status display button (away from desk button, eating button, etc.) or from another system 5 such as an attendance management system. Depending on the content of the online status, for example, an away from desk mark, a eating mark, a long working hours mark, an overtime warning mark, an offline duration mark, etc. are displayed around the avatar.

[0115] Information regarding the "floor status" is acquired from the presence or absence of avatars entering or leaving the virtual office 4, the presence or absence of other avatars within a predetermined range of the operating user's avatar, etc. When another avatar for which no seat GP22 is provided enters the virtual office 4, a door ping-pong sound is made to ring from the operating user's client terminal 3. When another avatar for which a seat GP22 is provided enters the virtual office 4, a sound effect of a door opening and closing is made to ring from the operating user's client terminal 3. When another avatar enters within a predetermined range of the operating user's avatar, a predetermined effect is executed as described above, and for example, a sound effect of footsteps is made to ring from the operating user's client terminal 3. In addition to footsteps, nearby avatars may be made to blink or to increase in size.

[0116] Information regarding the "calendar status" is obtained, for example, from schedule management software.

[0117] Information regarding the "communication state" is obtained from, for example, the usage state of video calls, the usage state of chat applications, and the like.

[0118] In addition, in the "control" mode, for example, when the user is in a meeting or on a phone call, it is possible to prohibit the start of a video call. Or, when the user is eating or traveling and another user requests a conversation, a message to that effect is sent to the client terminal 3 and displayed.

[0119] In the embodiment described below, the user's situation is determined by analyzing information acquired from a workflow or a TODO list.

[0120] 11 is an example of an interest information filter T2 used to extract user situation information. The information providing unit 24 of this embodiment extracts predetermined user situation information from each user situation information based on the interest information filter T2 set for each user, and provides the extracted user situation information to the virtual office 4 in association with an avatar operated by each user. That is, in this embodiment, the display content of the user situation information associated with the avatars of other users is different for each user.

[0121] The interest information filter T2 can also be called a "predetermined policy." The interest information filter T2 is prepared for each user. The interest information filter T2 specifies the status of other users to be provided to the corresponding user.

[0122] 11 includes, for example, a management number C21, a filter name C22, and a display content C23. The filter name C22 succinctly expresses the content of the interest information filter. The display content C23 is the content of the user situation information extracted by the interest information filter.

[0123] The user status information displayed for other avatars on the same floor, the user status information displayed for other avatars in the same group, the user status information displayed for other avatars in the same subgroup, and the user status information displayed for the avatar of a user specified by the operating user may each be different.

[0124] Here, a "group" is a group to which multiple users belong, and can be rephrased as a department. A "subgroup" is a small group that belongs to a group, and can be rephrased as a team.

[0125] For other avatars on the same floor, for example, "1. Business status" and "5. Calendar status" are displayed. For other avatars in the same group, for example, "1. Business status", "3. Online status", "4. Floor status", and "6. Communication status" are displayed. For other avatars in the same subgroup, for example, "1. Business status", "2. User status (status of the user in front of the client terminal)", "3. Online status", "4. Floor status", "5. Calendar status", and "6. Communication status" are displayed. For avatars of other users specified by the user, for example, "1. Business status", "5. Calendar status", and "6. Communication status" are displayed.

[0126] If another avatar (another user) corresponds to multiple filters of interest information filter T2, a filter corresponding to a lower concept may be applied. For example, if another avatar on the same floor belongs to a certain group, the "within the same group" filter with management number "2" is applied. If a user specifies an avatar of a specific user who belongs to a subgroup, the "specified user" filter with management number "4" is applied to the avatar of the specified user.

[0127] The present invention is not limited to the above example, and a filter with less display content may be applied preferentially, or conversely, a filter with more display content may be applied preferentially.

[0128] 11 controls the content displayed in each filter according to the avatar density in the virtual office 4. As the avatar density increases, the amount of user situation information placed around each avatar is reduced, allowing users to easily grasp the situations of other users.

[0129] According to the virtual space video calling system 1 of this embodiment configured as described above, a user can start a video call simply by touching an avatar representing the user to the avatar of the person with whom the user wants to talk. This allows the user to easily use video calling with intuitive operations, and allows for easy communication similar to that of a conversation in a real office.

[0130] Furthermore, in this embodiment, the user can easily select a user to be a video call partner from the follow list 250, improving the usability of video calling.

[0131] Furthermore, in this embodiment, since the user can immediately grasp the status of the candidate user from the user status displayed in the follow list 250, the user can easily determine whether or not to accept a video call.

[0132] Furthermore, in this embodiment, the user can also check the status of the avatar of the candidate user by operating the confirmation button 251. This allows the user to more appropriately determine whether or not to accept a video call.

[0133] In this embodiment, the user can make his / her own avatar appear near the candidate user's avatar by operating the movement button 252. Then, the user can start a video call by moving from the appearance location of his / her own avatar toward the candidate user's avatar and touching it.

[0134] In this embodiment, the status of the avatar corresponding to the candidate user for the video call can be checked on the spot without moving the avatar of the user who wants to make the video call, and after checking, the user's own avatar can be instantly moved close to the avatar of the candidate user to bring the avatars into contact with each other and start the video call. As a result, in this embodiment, in a virtual office 4 with a large one floor or a virtual office 4 with multiple floors, the availability of a video call can be appropriately determined, a video call can be easily started, and usability for the user is improved.

[0135] Furthermore, in this embodiment, the user's face image is displayed inside the avatar, so computer resources and network resources can be saved. Furthermore, in this embodiment, the user can know the appearance of the other party through the face image, and the face image does not cover the screen other than the avatar, so the parties in the video call can make the video call while checking the situation in the virtual office 4.

[0136] In this embodiment, a user who wishes to join an ongoing video call can join the video call simply by touching their own avatar to any of the avatars participating in the ongoing video call. EXAMPLES

[0137] The second embodiment will be described with reference to Figures 12 and 13. In the following embodiments including this embodiment, differences from the first embodiment will be mainly described.

[0138] In this embodiment, the number of other users viewing the user's avatar is displayed in the virtual office 4. Furthermore, in this embodiment, the number of viewing users is displayed according to the relationship between the viewing user and the user being viewed.

[0139] 12 shows a process for displaying a virtual office. This process is executed by the call candidate management unit 25 and the virtual office control unit 21. In the following, the server 2 will be described as the subject.

[0140] In the virtual office display process S37A similar to step S32 in Fig. 7, after arranging and displaying objects such as avatars and furniture in the virtual office 4 (S61), it is determined whether any user is viewing the user's avatar (S62). The server 2 can detect which user is viewing whose avatar by acquiring the operation state of the confirmation button 251 from each client terminal 3. Viewing an avatar here means that not only the avatar being viewed, but also the surroundings of that avatar are provided to other users.

[0141] When even one browsing user is detected (S62: YES), the server 2 detects the relationship between the browsing user and the user being viewed (S63). The relationship may be, for example, a mutual follow relationship in which both users are registered in the follow list 250, a relationship in which both users belong to the same department, a relationship in which both users belong to the same project, a relationship in which both users have a common follower, or any other relationship (other relationship).

[0142] As shown in Fig. 13, the server 2 displays the number of users currently viewing the item for each detected relationship in the viewing status window W3 (S64). In Fig. 13, the user is notified of "mutual follow relationships: 1, same department: 1, others: 1." In the case of mutual follow relationships, the specific effect GP20 may be used to indicate which user in the follow list 250 is viewing the item.

[0143] When the server 2 receives an instruction to change the status from the user (S65: YES), it changes the status displayed on the user's avatar (S66). There are statuses that can be used to indicate an intention to decline a video call, such as the above-mentioned on the phone, in a meeting, and away from the desk, and statuses that can be used to indicate an intention to make a video call, such as "want to chat."

[0144] The present embodiment thus configured achieves the same effects as those of the first embodiment. Furthermore, in this embodiment, the number of users currently viewing the user's avatar is displayed in the viewing status window W3, so that the user can know the degree of attention being given to him or her.

[0145] Furthermore, in this embodiment, the number of users who are viewing the document is displayed in the viewing status window W3 according to their relationship with the user, thereby improving user convenience.

[0146] Furthermore, in this embodiment, after checking the display contents of the browsing status window W3, the user can change the status displayed on his / her avatar that is directly or indirectly related to the video call, thereby allowing the user to express through the avatar whether to reject the video call in advance or whether to accept the video call passively or actively, thereby improving the user's convenience. EXAMPLES

[0147] The third embodiment will be described with reference to Fig. 14. Fig. 14 is a flow chart showing a process of analyzing a user's situation based on information acquired from the client terminal 3 and information acquired from the other system 5.

[0148] The process of Fig. 14 is a modified example of the process described in Fig. 6. The information collecting application 32 performs a primary analysis on the information collected from the user interface unit 33 and the like before transmitting the information to the server 2 (S16). The information collecting application 32 transmits the result of the primary analysis to the information collecting unit 23 (S15).

[0149] Here, the primary analysis means, for example, calculating the number of key inputs per unit time, or calculating the change in the user image per unit time (such as the movement trajectory of feature points set on the user image).

[0150] The information collection unit 23 of the server 2 receives information (primary analysis results) from each client terminal 3 (S21). Furthermore, the information collection unit 23 acquires the progress status of the workflow from the other systems 5 (S25) and also acquires the start status of the TODO list (S26).

[0151] The server 2 stores the information received in steps S21, S25, and S26 in the database 26 (S22), and performs a secondary analysis (S23). The secondary analysis involves judging the degree of busyness of the user from the number of keystrokes per unit time, estimating the user's actions from the movement trajectory of the user's image, and estimating the user's work status from the progress status of the workflow or the TODO list. These judgments or estimations can be realized by using artificial intelligence technology. For example, an artificial intelligence model can be generated by learning a set of the user's actual status and the progress status of the workflow as teacher data. If the progress status of the workflow is input to the artificial intelligence model, the user's work status related to the progress status of the workflow can be estimated.

[0152] The present embodiment thus configured also achieves the same effects as those of the first embodiment. Furthermore, in this embodiment, the analysis of user situation information is shared between the information collection application 32 and the information collection unit 23, thereby reducing the load on the server 2. Furthermore, in this embodiment, the user's work situation can be automatically estimated from the progress of the workflow, the TODO list, and the like, and the status can be controlled, eliminating the need for the user to manually change the status, improving usability for the user.

[0153] Note that the operation of the avatar by the user is not limited to manual operations such as dragging. For example, the user's line of sight may be detected, and the movement of the avatar may be controlled based on the detected line of sight. Alternatively, the user's body movements such as the movements of the hands and feet may be detected by a camera, and the avatar may be moved according to the user's body movements.

[0154] The present invention is not limited to the above-described embodiment. A person skilled in the art can make various additions and modifications within the scope of the present invention. The above-described embodiments can be appropriately combined and implemented. [Explanation of symbols]

[0155] 1: Virtual space video calling system, 2: Virtual office providing server, 3: Client terminal, 4: Virtual office, 21: Virtual office control unit, 22: Video call control unit, 23: Information gathering unit, 24: Information providing unit, 25: Call candidate management unit, GP1: Avatar

Claims

1. A virtual space video call system in which a plurality of users make a video call through a virtual space, a virtual space control unit that controls a virtual space provided to the client terminals used by the multiple users; a video call control unit that controls a video call in the virtual space; a call candidate management unit that manages a predetermined user among the plurality of users as a candidate for the video call; The call candidate management unit includes a user selection unit that displays information of a predetermined user and selects a candidate user who will be a video call partner from among the predetermined users, a candidate user status confirmation unit that displays information indicating the status of the candidate user, and a movement unit that moves an avatar of a user who wishes to make a video call within a predetermined range of the avatar of the candidate user selected by the candidate user selection unit. Virtual space video chat system.

2. When the video call control unit receives a predetermined instruction, the video call control unit starts a video call between a client terminal used by the user who desires to make a video call and a client terminal used by the selected candidate user. The virtual space video calling system according to claim 1 .

3. the virtual space control unit transmits the predetermined instruction to the video call control unit when it is determined that the avatars in the virtual space have come into contact with each other; The video call control unit starts a video call between client terminals used by users corresponding to the avatars determined to have come into contact by the virtual space control unit. The virtual space video calling system according to claim 2 .

4. The information of the predetermined user includes a user status indicating a status of the predetermined user, The candidate user status confirmation unit displays an avatar status indicating a surrounding status of the avatar of the candidate user. The virtual space video calling system according to claim 1 .

5. The virtual space control unit displays a predetermined effect indicating that the avatar of the user wishing to make a video call is near the avatar of the candidate user when the avatar of the user wishing to make a video call moves within the predetermined range. The virtual space video calling system according to claim 1 .

6. the virtual space has a plurality of virtual floors; The virtual floor where the avatar who wishes to make a video call is located is different from the virtual floor where the avatar of the candidate user is located. The virtual space video calling system according to claim 1 .

7. The call candidate management unit displays a number of viewers, which is the number of users currently viewing an avatar of the user who wishes to make a video call. The virtual space video calling system according to claim 1 .

8. The call candidate management unit displays the number of viewers according to the relationship between the user viewing the avatar of the user who wishes to make a video call and the user who wishes to make a video call. The virtual space video calling system according to claim 7.

9. The user situation is created based on information acquired from a management system that manages the user's work situation and information acquired from a client terminal used by the user. The virtual space video calling system according to claim 4.

10. The video call control unit starts the video call by displaying, in each of the avatars that have come into contact with each other in the virtual space, an image acquired from a client terminal used by a user corresponding to each of the avatars. The virtual space video calling system according to any one of claims 3 to 9.

11. A virtual space video calling method in which client terminals used by multiple users make a video call through a virtual space, comprising: A computer connected to the plurality of client terminals so as to be capable of two-way communication includes: A virtual space control unit controls the virtual space provided to the client terminals used by the multiple users; A video call control unit controls the video call in the virtual space; managing a predetermined user among the plurality of users as a candidate for a video call; displaying information of the predetermined user, and having a user selection unit select candidate users to be video call counterparts from among the predetermined users; displaying information indicating the candidate user's status by a candidate user status confirmation unit; a moving unit moving an avatar of a user who desires to make a video call within a predetermined range of an avatar of a candidate user selected by the candidate user selection unit; determining by the virtual space control unit whether avatars in the virtual space have come into contact with each other; The video call control unit starts a video call between the client terminals used by the users corresponding to the avatars determined to have come into contact by the virtual space control unit. How to make a virtual space video call.

Images