System and control method for the same

Abstract

To solve the problem in which there were cases where a user avatar performed unintended actions.SOLUTION: A virtual space management system 111 executes control for restricting avatar actions based on detection data of a user corresponding to an avatar on the basis of either an action restriction mode, which restricts the action of the avatar, or an input restriction mode, which restricts user input. (S510, S611 to S616 in Figure 6(b), S811 to S816 in Figure 8, S1001 to S1006 in Figure 10(a), and S1011 to S1016 in Figure 10(b)).SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a system for controlling the movement of an avatar in a virtual space and a method for controlling the system. [Background technology]

[0002] XR (Extended Reality / Cross Reality) is a general term for technologies that create spaces that provide simulated experiences by fusing the real and virtual worlds, such as VR (Virtual Reality) and AR (Augumented Reality), and various standardization efforts are underway. In recent years, virtual spaces and services that use such technologies are called metaverses, and they are being used not only for entertainment purposes such as games, but also in business situations, such as virtual offices and VR conference rooms.

[0003] In the virtual space, each user wears an HMD (Head Mounted Display) and communicates with other users' avatars in the virtual space via the HMD just as they would in the real world. The HMD tracks the position and gestures of the user's body, hands, HMD and controller, allowing the user to move an avatar (user avatar) in the virtual space in the same way as the user moves in real space.

[0004] A technology for recognizing a user's gestures and manipulating a virtual object is disclosed in Patent Document 1. According to Patent Document 1, by detecting and analyzing a user's gestures, it is possible to input the gesture information into an application and operate a virtual object, thereby improving the user experience in manipulating virtual objects. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Special Publication No. 2012-521039 Summary of the Invention [Problem to be solved by the invention]

[0006] However, if a user's gestures are always input to an application and reflected in the user's avatar in the virtual space, the user's avatar may move even when the user does not want it to. For example, even if the user does not want the user's avatar to move during a conversation in the virtual space, if the user changes their posture in the real world due to fatigue or other reasons, for example, by sitting down, the user's avatar will also sit down.

[0007] In addition, the above-mentioned gesture detection may include false positives, so a gesture different from the one the user performed may be input. For example, when a user raises their arm in the real world to scratch their head, it may be falsely detected as a sweeping motion, and the user avatar's arm may hit the user avatar of a nearby user.

[0008] The present invention has been made to solve the above-mentioned problems, and has an object to provide a mechanism for preventing a user avatar from performing actions that are not intended by the user. [Means for solving the problem]

[0009] The present invention is a system for controlling the movement of an avatar in a virtual space, and is characterized by having a control means that executes control to restrict the movement of the avatar based on detection data of a user corresponding to the avatar, based on either a movement restriction mode that restricts the movement of the avatar, or an input restriction mode that restricts user input. Effect of the Invention

[0010] According to the present invention, it is possible to prevent a user avatar from performing actions that are not intended by the user. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of the overall configuration of a system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a hardware configuration diagram of a server computer and a client terminal that constitute the system according to the embodiment. [Diagram 3] FIG. 2 is a software configuration diagram of the system according to the embodiment. [Figure 4A] FIG. 4 is a diagram showing a display screen of a client terminal according to the first embodiment. [Figure 4B] FIG. 4 is a diagram showing a display screen of a client terminal according to the first embodiment. [Diagram 5] FIG. 4 is a diagram showing a sequence of the system according to the embodiment. [Figure 6] 5 is a flowchart illustrating a control information setting process and a user avatar operation process in the first embodiment. [Figure 7] FIG. 11 is a diagram showing a display screen of a client terminal according to the second embodiment. [Figure 8] 10 is a flowchart illustrating a user avatar operation process in the second embodiment. [Figure 9A] FIG. 13 is a diagram showing a display screen of a client terminal according to the third embodiment. [Figure 9B] FIG. 13 is a diagram showing a display screen of a client terminal according to the third embodiment. [Figure 10] 13 is a flowchart illustrating a user avatar operation process according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. [First embodiment] FIG. 1 is a diagram illustrating an example of the overall configuration of a system according to an embodiment of the present invention. 1, in the system of this embodiment, a virtual space management system 111 and client terminals 121 and 132 are connected via networks 101 to 103. In addition, the client terminal 131 is connected to a client terminal 132.

[0013] The networks 101 to 103 are so-called communication networks realized by, for example, a LAN such as the Internet, a WAN, a telephone line, a dedicated digital line, an ATM or frame relay line, a cable television line, a wireless line for data broadcasting, etc. The networks 101 to 103 may be wired or wireless as long as they are capable of transmitting and receiving data. In this embodiment, the network 101 is the Internet, and the networks 102 and 103 are wireless LANs set up on the Internet or in general household or company networks, etc.

[0014] The client terminals 121 and 131 are, for example, dedicated hardware compatible with drawing virtual objects handled in XR, such as an HMD (head mounted display) or smart glasses, or a mobile terminal with a built-in program execution environment, such as a smartphone. The client terminals 121 and 131 also include a camera for capturing images of the surroundings and a display for displaying virtual objects. The client terminals 121 and 131 provide a simulated experience that combines the real world and the virtual world by recognizing the user's fingers through the camera and superimposing actions in the real space and actions in the virtual space. If the client terminals 121 and 131 are not dedicated hardware, such as a smartphone, they use an API (Application Programming Interface) provided by a web browser, an OS (Operating System), or the like to draw virtual objects. In this embodiment, the client terminals 121 and 131 are described as HMDs.

[0015] The client terminal 132 is a PC (Personal Computer), and is an information processing device having a display such as a notebook PC, a desktop PC, a smartphone, etc. The HMD may perform processing independently like the client terminal 121, or may perform processing in conjunction with a PC like the client terminals 131 and 132.

[0016] The virtual space management system 111 is a system for providing each client terminal with virtual objects in a virtual space, avatars that mimic each user, virtual objects provided by services and applications, and their operation information. The virtual space management system 111 also manages users who use the client terminals 121, 122, and 132. That is, the virtual space management system 111 receives login and logout requests from the client terminals 121 and 132 and performs login and logout processing. The virtual space management system 111 is constructed using a server computer. Alternatively, the virtual space management system 111 can be constructed by adopting cloud computing technology or the like. Hereinafter, the avatar of a user that operates in a virtual space is called a "user avatar."

[0017] The functions of the virtual space management system 111 described in this embodiment may be realized by a single server or a single virtual server, or may be realized by multiple servers or multiple virtual servers, or multiple virtual servers may be executed on a single server.

[0018] FIG. 2 is a diagram showing an example of the hardware configuration of the virtual space management system 111 and the client terminals 121, 131, and 132. As shown in FIG. 2, a CPU (Central Processing Unit) 202 controls the entire device. The CPU 202 executes application programs, an OS, and the like stored in a HDD 205, and controls temporary storage of information, files, and the like required for executing the programs in a RAM 203.

[0019] A GPU (Graphics Processing Unit) 210 performs calculations necessary for rendering virtual objects and avatars in real time. A ROM (Read Only Memory) 204 is a storage unit, and stores various data such as basic I / O programs. A RAM (Random Access Memory) 203 is a temporary storage unit, and functions as a main memory, a work area, etc. for the CPU 202 and the GPU 210 .

[0020] The HDD (Hard Disk Drive) 205 is one of the external storage units, functions as a large-capacity memory, and stores application programs such as a Web browser, programs for a group of service servers, an OS, related programs, etc. Note that the configuration may include other storage devices such as an SSD (Solid State Drive) or an eMMC (embedded MultiMediaCard) instead of or in combination with the HDD.

[0021] The display 206 is a display unit that displays virtual objects, information necessary for operations, etc. The display 206 may also be a device that has a function of receiving operation instructions from a user, such as a touch panel. The camera 207 may be an out-camera in the client terminal 121, 131 that captures images of the surroundings, or an in-camera that mainly captures images of the user himself / herself. By analyzing images captured by the camera 207, particularly the out-camera, with a program stored in the HDD 205, it is possible to synchronize the actions of the user in the real space with the actions of the user avatar in the virtual space. The virtual space management system 111 and the client terminal 132 do not necessarily require the camera 207.

[0022] The interface 208 is an external device I / F that connects peripheral devices such as various external sensors. It is possible to operate virtual objects in a virtual space by recognizing the user's movements in the real space with the camera 207 described above, but the same function can also be achieved by operating a dedicated controller connected to the interface 208.

[0023] The speaker 209 is a device that converts electrical signals in the client terminals 121, 131, and 132 into physical sound, and the user can hear the sounds provided by the virtual space management system 111 and the client terminals 121, 131, and 132 through the speaker 209. The user may hear the sounds through the speaker 209, or through an external device such as headphones connected to the interface 208.

[0024] The system bus 201 controls the flow of data within the device. A NIC (Network Interface Card) 211 exchanges data with external devices via the networks 100-102.

[0025] The above computer configuration is an example, and is not limited to the configuration example in Fig. 2. For example, the storage destination of data and programs can be changed to RAM 203, ROM 204, HDD 205, etc. depending on the characteristics of the data and programs. In addition, the CPU 202 and GPU 210 execute processing based on the programs stored in HDD 205, thereby realizing functions and processing in the software configuration shown in Fig. 3.

[0026] FIG. 3 is a diagram for explaining the software configuration of the virtual space management system 111 and the client terminals 121, 131, and 132, and particularly shows the software configuration from which functions related to this embodiment are extracted.

[0027] The virtual space management system 111 includes, as user management functions, a user management unit 301 and a login processing unit 302. The virtual space management system 111 also includes, as basic functions for providing a virtual space, a virtual object management unit 303, a virtual object providing unit 304, a virtual object control information acquisition unit 305, a virtual object control information management unit 306, and an operation information acquisition unit 307.

[0028] The user management unit 301 manages user information and login information. The login processing unit 302 receives login requests from the client terminals 121, 131, and 132, collates the information with the user management unit 301, and returns the login processing result to the client terminals 121, 131, and 132.

[0029] Table 1 below shows an example of data managed by the user management unit 301. [Table 1]

[0030] The user ID column is an ID that uniquely identifies a user. The password column is the basic authentication password used when logging in with the user ID. The login processing unit 302 checks the combination of user ID and password included in the login request from the client terminals 121, 131, and 132 against Table 1, and if there is a match, returns the login result to each client terminal as successful. The login status column indicates the login status of the user, with on indicating a logged-in state and off indicating a logged-out state. The login expiration column indicates the expiration date of the authentication status of the logged-in user.

[0031] The virtual object management unit 303 manages virtual objects placed in the virtual space, 3D data of each user avatar, and the like. The virtual object providing unit 304 provides the 3D data managed by the virtual object management unit 303 to the client terminals 121 , 131 , and 132 .

[0032] Table 2 below shows an example of data of each virtual object managed by the virtual object management unit 303. [Table 2]

[0033] The virtual object ID column is an ID that uniquely identifies a virtual object in the virtual space. The 3D data column is data of 3D models in various formats. The user column indicates which user's avatar the virtual object is, and if it is blank, it indicates that it is not a user avatar but a virtual object prepared by an application or the like. The action control column indicates whether the action of the virtual object is controlled, and if it is true, it indicates that the action is controlled, and if it is false, it indicates that the action is not controlled. The content of the action control is managed by the virtual object control information management unit 306 described later.

[0034] The virtual object control information acquisition unit 305 acquires control information related to virtual objects from the client terminals 121 , 131 , and 132 . The virtual object control information management unit 306 manages the control information acquired by the virtual object control information acquisition unit 305 .

[0035] Table 3 below shows an example of data of control information for each virtual object managed by the virtual object control information management unit 306. [Table 3]

[0036] The virtual object ID column is the same as that explained in Table 2, and virtual object IDs for which the action control column in Table 2 is true are managed. The action restriction range column indicates the range of the virtual object for which action is restricted. In Table 3, as an example, the coordinates of the 3D model of the virtual object are used as values, and the range specified by these coordinates indicates that the virtual object will not move even if an operation input is performed. In Table 3, the range is specified by eight coordinates as an example, but the number of coordinates does not have to be eight, and the range may be specified by other means rather than by coordinates.

[0037] The operation information acquisition unit 307 acquires operation information related to virtual objects from the client terminals 121, 131, and 132. The virtual space management system 111 determines the operation of the virtual object based on the operation information acquired by the operation information acquisition unit 307 and the information managed by the virtual object management unit 303 and the virtual object control information management unit 306.

[0038] For example, a case where operation information of a virtual object whose virtual object ID is the userA avatar is acquired will be described using Tables 2 and 3 as examples. The virtual space management system 111 acquires information about the userA avatar (illustrated in Table 2) from the virtual object management unit 303. In the example of Table 2, the action control column of the virtual object of the userA avatar is true and is subject to action control, so the virtual space management system 111 acquires control information about the userA avatar (illustrated in Table 3) from the virtual object control information management unit 306. Then, based on the action restriction range of the userA avatar contained in the acquired restriction information and the operation information acquired by the operation information acquisition unit 307, the operation of the virtual object is determined so that the range of the virtual object that does not fall within the action restriction range is operated and the range of the virtual object that falls within the action restriction range is not operated.

[0039] Next, the software configuration of the client terminals 121, 131, and 132 will be described. The client terminals 121, 131, and 132 each include an input information acquisition unit 320, an input information analysis unit 321, a display unit 322, a login unit 323, a local virtual object management unit 324, and a virtual object acquisition unit 325. The client terminals 121, 131, and 132 further include a virtual object control information provision unit 326, a virtual object control information management unit 327, and an operation information provision unit 328.

[0040] The input information acquisition unit 320 acquires information such as an image of a finger captured by the camera 207 or information input by an input device connected to the interface 208 . The input information analysis unit 321 analyzes user actions such as gestures acquired by the input information acquisition unit 320, and converts them into input information that can be handled by the system or applications. The display unit 322 displays virtual objects and user avatars in the virtual space via the display 206 .

[0041] The login unit 323 transmits the user name and password entered by the user to the login processing unit 302. Note that the authentication method for the virtual space management system 111 may be face authentication using a face image captured by the camera 207, iris authentication using the iris, or biometric authentication such as fingerprint authentication using a fingerprint sensor connected to the interface 208.

[0042] The local virtual object management unit 324 manages information such as virtual objects and user avatar 3D data and the actions of the 3D data obtained from the virtual space management system 111 on the client terminals 121 , 131 , and 132 . The virtual object acquisition unit 325 acquires information such as 3D data of virtual objects and user avatars and the operation of the 3D data from the virtual object provision unit 304 , and stores the information in the local virtual object management unit 324 .

[0043] The virtual object control information management unit 327 manages the virtual object restriction information input by the user. The information managed here is the same as the information shown in the virtual object control information management unit table of Table 3. For example, when userA operates a client terminal, only the information of userA avatar, which is the user avatar of userA, is managed.

[0044] The virtual object control information providing unit 326 transmits the information managed by the virtual object control information managing unit 327 to the virtual object control information acquiring unit 305 . The operation information providing unit 328 transmits the user avatar operation information acquired by the input information acquiring unit 320 and analyzed by the input information analyzing unit 321 to the operation information acquiring unit 307 .

[0045] The above-described functions 301 to 328 make it possible to provide the user with real-time operations of virtual objects and user avatars in the virtual space provided by the virtual space management system 111. In addition, it is possible to control the user avatar so that it does not perform operations that are not intended by the user.

[0046] A method for controlling the movement of a user avatar in a virtual space, which is an object of the present invention, will be described with reference to Figures 4A, 4B, 5, and 6. Hereinafter, Figures 4A and 4B will be collectively referred to as "Figure 4."

[0047] FIG. 4(a) is a diagram showing an example of an image displayed on the display 206 of the client terminal 121, 131, and shows an example of an input UI for controlling the operation of a user avatar in this embodiment.

[0048] 4(b) to 4(c) are diagrams showing examples of user gestures and the actions of the user avatar resulting from the gestures, and in particular show an action example in which the user makes a gesture of raising the left arm. In detail, FIG. 4(b) shows a user in the real world who uses the client terminal 121, and illustrates an example of the user's gesture in this embodiment. Figures 4(c) and 4(d) are examples of images in a virtual space displayed on the display 206 of the client terminal 131, and show examples of the movements of a user avatar in this embodiment. Note that Figure 4(c) corresponds to an example of the movements when movement control is not set, and Figure 4(d) corresponds to an example of the movements when movement control is set.

[0049] In this embodiment, it is assumed that the user operating client terminal 121 is the user whose user ID column in Tables 1 and 2 described above is userA. Also, it is assumed that Fig. 4(a) is the image displayed on display 206 of client terminal 121 of userA, and Fig. 4(b) is the user of userA. On the other hand, it is assumed that the user operating client terminals 131 and 132 is the user whose user ID column in Tables 1 and 2 described above is userB. Also, it is assumed that Figs. 4(c) and 4(d) are the images displayed on display 206 of client terminal 131 of userB.

[0050] First, FIG. 4(a) will be described. FIG. 4(a) is an image from the viewpoint of a user whose user ID is userA managed in the user management table of Table 1, and shows an example of an input UI for controlling the operation of a user avatar in this embodiment.

[0051] Reference numeral 401 denotes the display 206 of the client terminal 121 used by userA. Reference numeral 411 denotes a user avatar of userA, which is managed by the virtual object management unit 303. In this example, it is assumed that the virtual object ID in the virtual object management table of Table 2 is the virtual object of the user avatar of userA avatar.

[0052] Numerals 421 to 425 denote multiple-choice input forms showing patterns for controlling operations. 421 to 424 are options for restricting the movement, and 425 is an option for releasing the restriction on the movement. 431 to 435 are range display frames showing the range of the movement restriction of the user avatar. 421 is an option for restricting the movement of the entire body of the user avatar, and the range display frame 431 selects the range of the entire body of the user avatar. 422 is an option for restricting the movement of the upper body of the user avatar, and the range display frame 432 selects the range of the upper body of the user avatar. 423 is an option for restricting the movement of the face of the user avatar, and the range display frame 433 selects the range of the face of the user avatar. 424 is an option for restricting the movement of the fingers of the user avatar, and the range display frames 434 and 435 select the range of the fingers of the user avatar. The user can restrict the movement of the avatar by selecting 421 to 424. Also, the user can release the movement restriction of the avatar by selecting 425.

[0053] Next, FIG. 4(b) to FIG. 4(c) will be described. FIG. 4(b) is an image in the real world of a user with a user ID of userA managed in the user management table of Table 1, and shows an example of a gesture made by the user in this embodiment. Reference numeral 451 denotes an appearance of userA in the real world, and reference numeral 452 denotes the client terminal 121, indicating that userA is using the client terminal 121. Reference numeral 453 denotes the left arm of userA, indicating that userA is making a gesture of raising his left arm.

[0054] Figure 4(c) shows an image in the virtual space when the userA avatar is viewed from the perspective of a user with a user ID of userB, who is managed in the user management table of Table 1, and this is the image when no motion control is set for the userA avatar. Reference numeral 461 denotes the display 206 of the client terminal 131 used by userB. Reference numeral 462 denotes the fingers of the user avatar of user B, and the virtual object ID in the virtual object management table of Table 2 indicates the virtual object of the user B avatar. Reference numeral 471 denotes a virtual object of the user avatar of user A, and reference numeral 472 denotes the left arm of the user avatar of user A. In FIG. 4(b), the left arm 472 of the user avatar is raised, similar to the left arm 453 of the user in the real world, because motion control is not set.

[0055] FIG. 4(d), like FIG. 4(c), is an image in the virtual space when userA's avatar is viewed from userB's user's perspective, where no motion control is set for userA's avatar. 461, 462, and 471 are the same as those in the explanation of Fig. 4(c). 473 indicates the left arm of the virtual object of the user avatar of user A. When motion control such as the full body restriction 421 or the upper body restriction 422 is set, the left arm of the user avatar is within the restricted range, so even if the left arm 453 of the user in the real world is raised, the left arm 472 of the user avatar is not raised and is not moving.

[0056] Next, the overall processing sequence will be described with reference to FIG. Fig. 5 is a diagram showing an example of a sequence in which a user controls or releases control over the action of a user avatar and shares the action of the user avatar with other users. The processing of the client terminal 121 shown in Fig. 5 is executed by the software configuration of the client terminal 121 shown in Fig. 3. Moreover, the processing of the virtual space management system 111 is executed by the software configuration of the virtual space management system 111 shown in Fig. 3. Moreover, the processing of the client terminals 131 and 132 is executed by the software configuration of the client terminals 131 and 132 shown in Fig. 3.

[0057] In S 501 , the login unit 323 of the client terminal 121 of user A transmits the user ID and password to the login processing unit 302 . In S502, the login processing unit 302 refers to the user management table of Table 1 managed by the user management unit 301, confirms that the user ID and password match those of userA, and if they match, returns a login result indicating that the login was successful. Similarly, in S 503 , the login unit 323 of the client terminals 131 and 132 of user B transmits the user ID and password to the login processing unit 302 . In S504, the login processing unit 302 refers to the user management table of Table 1 managed by the user management unit 301, confirms that the managed user ID and password match those of userB, and if they match, returns the login result indicating that the login was successful.

[0058] Thereafter, in S505 to S520, steps related to setting virtual object operation control, operating the user avatar, the operation of the user avatar, and sharing of the user avatar shown in FIG. 4 are executed, and are executed periodically and asynchronously with other steps.

[0059] In S505, the virtual object control information management unit 327 of the client terminal 121 of user A sets the control information of the virtual object. Here, the specific processing content of S505 will be described with reference to the flowchart in FIG.

[0060] FIG. 6(a) is a flowchart showing an example of a control information setting process, which is executed by the software configuration of the client terminal 121 shown in FIG. In S601, when the input information acquisition unit 320 and the input information analysis unit 321 acquire a control information change request input from the user, the process proceeds to S602. In S602, the input information acquisition unit 320 acquires control information input by the user. The control information change request may be a change request initiated by the user, or may be a change request initiated by the system or an application.

[0061] Next, in S603, the input information analysis unit 321 checks whether the control information acquired in S602 above is control release information. If it is control release information (Yes in S603), the input information analysis unit 321 advances the process to S604. In S604, the input information analysis unit 321 deletes the control information already stored in the virtual object control information management unit 327, and ends the processing of this flowchart.

[0062] On the other hand, if the control information acquired in S602 is not control release information (No in S603), the input information analysis unit 321 advances the process to S605. In S605, the input information analysis unit 321 stores the control information acquired in S602 in the virtual object control information management unit 327, and ends the processing of this flowchart.

[0063] Returning now to the explanation of the flowchart in FIG. Next, in S506 , the virtual object control information providing unit 326 of the client terminal 121 of user A transmits the control information stored in the virtual object control information managing unit 327 to the virtual object control information acquiring unit 305 . In S507, upon receiving the control information, the virtual object control information acquisition unit 305 returns a notification to the virtual object control information provision unit 326 indicating that reception has been completed.

[0064] In S508, the operation information providing unit 328 of the client terminal 121 of userA transmits to the operation information acquiring unit 307 the user avatar operation information from the user acquired by the input information acquiring unit 320 and the input information analyzing unit 321. In S509, upon receiving the user avatar operation information, the operation information acquisition unit 307 returns a notification to the operation information provision unit 328 that reception has been completed.

[0065] Next, in S510, the virtual object management unit 303 operates the user avatar based on the virtual object control setting received in S506 and the user avatar operation information received in S508. Here, the specific processing content of S510 will be described with reference to the flowchart in FIG.

[0066] FIG. 6(b) is a flowchart showing an example of a user avatar operation process, which is executed by the software configuration of the virtual space management system 111 shown in FIG. In S611, the virtual object management unit 303 acquires information on the user avatar that is the operation target of the user avatar operation information received by the operation information acquisition unit 307 in S508 from the virtual object management unit 303, and determines whether operation control is set.

[0067] Here, if no action control is set for the user avatar (No in S611), the virtual object management unit 303 advances the process to S613. In S613, since no operation control is set for the user avatar, the virtual object management unit 303 causes the user avatar to operate in accordance with the user avatar operation information received by the operation information acquisition unit 307 in S508 above. Then, the process of this flowchart ends.

[0068] On the other hand, if action control is set for the user avatar (Yes in S611), the virtual object management unit 303 advances the process to S612. In S612 , the virtual object management unit 303 acquires the action control information set for the user avatar from the virtual object control information management unit 306 . Next, in S614, the virtual object management unit 303 determines whether the motion restriction range of the motion control information acquired in the above S612 is included in the user avatar operation information received by the operation information acquisition unit 307 in the above S508.

[0069] Here, if the operation restriction range of the operation control information acquired in S612 above is included in the user avatar operation information received by the operation information acquisition unit 307 in S508 above (Yes in S614), the virtual object management unit 303 proceeds to S615. In S615, the virtual object management unit 303 does not operate the user avatar within the operation restriction range of the operation control information acquired in S612 among the user avatar operation information received by the operation information acquisition unit 307 in S508, and operates the user avatar according to the user avatar operation information outside the operation restriction range. Then, the processing of this flowchart ends.

[0070] On the other hand, if the operation restriction range of the operation control information acquired in S612 above is not included in the user avatar operation information received by the operation information acquisition unit 307 in S508 above (No in S614), the virtual object management unit 303 proceeds to S616. In S616, the virtual object management unit 303 moves the user avatar in accordance with the user avatar operation information, and then ends the processing of this flowchart.

[0071] Returning now to the explanation of the flowchart in FIG. In S511, the virtual object acquisition unit 325 of the client terminals 131 and 132 of userB transmits a user avatar acquisition request, which is a virtual object, to the virtual object providing unit 304. In S512, upon receiving the user avatar acquisition request, the virtual object providing unit 304 transmits the virtual object to the virtual object acquiring unit 325. As a result, the virtual object acquiring unit 325 acquires the virtual object.

[0072] In S513, the virtual object control information management unit 327 of the client terminals 131 and 132 of user B sets the control information of the virtual object. Here, the processing content of S513 is the same as that of S505, so a description thereof will be omitted.

[0073] In S514 , the virtual object control information providing unit 326 of the client terminals 131 and 132 of user B transmits the control information stored in the virtual object control information managing unit 327 to the virtual object control information acquiring unit 305 . In S515, upon receiving the control information, the virtual object control information acquisition unit 305 returns a notification to the effect that reception has been completed to the virtual object control information provision unit 326.

[0074] In S516, the operation information providing unit 328 of the client terminals 131 and 132 of userB transmits to the operation information acquiring unit 307 the user avatar operation information from the user acquired by the input information acquiring unit 320 and the input information analyzing unit 321. In S517, upon receiving the user avatar operation information, the operation information acquiring unit 307 returns to the operation information providing unit 328 a notification that reception has been completed.

[0075] Next, in S518, the virtual object management unit 303 operates the user avatar based on the virtual object control setting received in S514 and the user avatar operation information received in S516. Here, the processing content of S518 is the same as that of S510, so a description thereof will be omitted.

[0076] In S519, the virtual object acquisition unit 325 of the client terminal 121 of user A sends a request to acquire a user avatar, which is a virtual object, to the virtual object providing unit 304. In S520, upon receiving the user avatar acquisition request, the virtual object providing unit 304 transmits the virtual object to the virtual object acquiring unit 325. As a result, the virtual object acquiring unit 325 acquires the virtual object.

[0077] As described above, in the first embodiment, a method for setting the movement control of a user avatar in a virtual space and a method for sharing the movement of the user avatar that has been movement controlled with other users have been described. With this method, the user avatar can be controlled so that it does not move or so that part of the user avatar does not move when the user does not wish, enabling communication with other user avatars.

[0078] Second Embodiment In the first embodiment, a method for setting a movement restriction range for a virtual object and controlling its movement as shown in the virtual object control information management unit table of Table 3 has been described. However, instead of setting a movement restriction range for a virtual object, a method for setting an input restriction range for a user and controlling its movement can also be considered. In the second embodiment, a method of setting an input limit range for a user and controlling the operation will be described.

[0079] Table 4 below shows an example of data of each virtual object managed by the virtual object management unit 303 in the second embodiment. [Table 4]

[0080] The virtual object ID column, 3D data column, and user column are the same as those described in Table 2. The input control column indicates whether or not the input of the user who operates the virtual object is controlled, and if it is true, it indicates that the input is controlled, and if it is false, it indicates that the input is not controlled. The contents of the input control are managed by a virtual object control information management unit 306 in the second embodiment described later.

[0081] Table 5 below shows an example of data of control information for each virtual object managed by the virtual object control information management unit 306 in the second embodiment. [Table 5]

[0082] The user ID column indicates the user ID for which the motion control is set, as explained in Table 1. The input restriction range column indicates the range of the user's body for which input restriction is set. In Table 5, as an example, the user's body part is used as a value, and even if an operation input is made with that body part, no operation will be performed. In Table 5, the input restriction range is specified by a body part as an example, but it may be specified by the body coordinates as in Table 3, or by other means.

[0083] A method of setting an input limit range for a user and controlling an operation in a second embodiment will be described with reference to FIG. 7 in addition to the diagrams of the first embodiment. FIG. 7 is a diagram showing an example of an input UI for controlling the operation of a user avatar in the second embodiment, and corresponds in particular to an image from the user's viewpoint whose user ID is userA managed in the user management table of Table 1.

[0084] Reference numeral 701 denotes the display 206 of the client terminal 121 used by userA. 711 shows a user in the real world. Numerals 721 to 725 denote multiple-choice input forms showing patterns for controlling operations, 721 to 724 denote options for restricting input, and 725 denote an option for releasing the control of operations. Numerals 731 to 735 denote range display frames showing the range of user input restrictions. Reference numeral 721 is an option for restricting input of the user's entire body, and range display frame 431 selects the range of the user's entire body. Reference numeral 722 is an option for restricting input of the user's upper body, and range display frame 732 selects the range of the user's upper body. Reference numeral 723 is an option for restricting input of the user's face, and range display frame 733 selects the range of the user's face. Reference numeral 724 is an option for restricting input of the user's fingers, and range display frames 734 and 735 select the range of the user's fingers. The user can restrict the user's input by selecting 721 to 724. The user can release the user's input restriction by selecting 725.

[0085] Next, the overall processing sequence will be described with reference to FIGS. The overall processing sequence of the second embodiment is similar to that of the first embodiment, but differs in the processing in S510 and S518 in which the virtual object management unit 303 operates the user avatar. Here, the specific processing content of S510 in the second embodiment will be described with reference to the flowchart in FIG.

[0086] FIG. 8 is a flowchart showing an example of a user avatar operation process in the second embodiment, which is executed by the software configuration of the virtual space management system 111 shown in FIG. In S811, the virtual object management unit 303 acquires information about the user avatar that is the target of the user avatar operation information received by the operation information acquisition unit 307 in S508 of Figure 5 from the virtual object management unit 303, and determines whether input control is set for the user performing the operation.

[0087] Here, if input control has not been set for the user performing the operation (No in S811), the virtual object management unit 303 advances the process to S813. In S813, since no input control is set for the user, the virtual object management unit 303 operates the user avatar in accordance with the user avatar operation information received by the operation information acquisition unit 307 in S508 above. Then, the process of this flowchart ends.

[0088] On the other hand, if input control is set for the user performing the operation (Yes in S811), the virtual object management unit 303 advances the process to S814. In S814, the virtual object management unit 303 determines whether the user avatar operation information received by the operation information acquisition unit 307 in S508 above includes the input restriction range of the input control information acquired in S812 above.

[0089] Here, if the user avatar operation information received by the operation information acquisition unit 307 in S508 above includes the input restriction range of the input control information acquired in S812 above (Yes in S814), the virtual object management unit 303 proceeds to S815. In S815, the virtual object management unit 303 does not operate the user avatar according to operation information that is within the input restriction range of the input control information acquired in S812 among the user avatar operation information received by the operation information acquisition unit 307 in S508, but operates the user avatar according to operation information that is not within the input restriction range. Then, the process of this flowchart ends.

[0090] On the other hand, if the user avatar operation information received by the operation information acquisition unit 307 in S508 does not include the input restriction range of the input control information acquired in S812 (No in S814), the virtual object management unit 303 proceeds to S816. In S816, the virtual object management unit 303 moves the user avatar in accordance with the user avatar operation information, and then ends the processing of this flowchart. Incidentally, the process of S518 in the second embodiment is similar to the process of S510 described above, and therefore a description thereof will be omitted.

[0091] As described above, an input restriction range can be set for a user, and the movement of the user avatar can be controlled based on this. With this configuration, the user avatar can be controlled so that it does not move or only part of the user avatar does not move when the user does not want to, allowing communication with other user avatars.

[0092] Third Embodiment In the above first and second embodiments, a method for setting a motion restriction range and an input restriction range and controlling a user avatar based on the range has been described, but there are cases where it is desired to restrict not only the range but also a specific shape of motion. For example, it may be desired to prevent the user avatar from reflecting the hand sign of raising the index finger for inputting information to the system, or to prevent the user avatar from making poses that may be considered inappropriate in terms of manners, such as a fist pump. In the third embodiment, a method of controlling the movement so as to restrict the movement of a specific shape will be described. In particular, the case where the movement of a virtual object is restricted as in the first embodiment and the case where an input restriction is imposed on a user as in the second embodiment will be described.

[0093] First, a case where a motion restriction is imposed on a virtual object as in the first embodiment will be described. Table 6 below shows an example of data of control information for each virtual object managed by the virtual object management unit 303 of the third embodiment when a motion restriction is imposed on the virtual object. [Table 6]

[0094] The Virtual Object ID column is the same as described in Table 3. The movement restriction shape column indicates the shape of a virtual object for which movement is restricted. In Table 6, as an example, the points of a 3D model of a virtual object and the coordinate range of those points are the values, and when a combination of a point and coordinate of the 3D model matches the value of the movement shape column and the virtual object takes a specific shape, it indicates that the virtual object will not move. In Table 6, the movement restriction shape is specified by six points and coordinate ranges as an example, but the number of points and coordinate ranges does not have to be eight, and the shape may be specified by other means rather than by points and coordinate ranges.

[0095] FIG. 9A is a diagram showing an example of an input UI for controlling the movement of a user avatar when imposing movement restrictions on a virtual object in the third embodiment, and corresponds to an image from the user's perspective for userA, whose user ID is managed in the user management table of Table 1.

[0096] Reference numeral 901 denotes the display 206 of the client terminal 121 used by userA. Reference numeral 911 denotes the hand of the user avatar of userA managed by the virtual object management unit 303, and shows a shape in which the index finger is pointed up. Reference numeral 912 denotes the arm of the user avatar of userA managed by the virtual object management unit 303, and shows the shape of a fist pump. Numerals 921 to 923 are multiple-choice input forms showing patterns for controlling movements, with 921 and 922 being options for restricting movements, and 923 being an option for releasing the movement control. By selecting 921 or 922, the user can impose a specific type of movement restriction on the avatar. Also, by selecting 923, the user can release the movement restriction on the avatar.

[0097] The overall processing sequence of the third embodiment is similar to that of the first embodiment, but differs in the processing in which the virtual object management unit 303 operates the user avatar in S510 and S518. Here, the specific processing content of S510 in the third embodiment will be described with reference to the flowchart in FIG.

[0098] FIG. 10(a) is a flowchart showing an example of a user avatar operation process when restricting the operation of a virtual object in the third embodiment, and is executed by the software configuration of the virtual space management system 111 shown in FIG. In S1001, the virtual object management unit 303 acquires information about the user avatar that is the operation target of the user avatar operation information received by the operation information acquisition unit 307 in S508 of Figure 5 from the virtual object management unit 303, and determines whether operation control is set.

[0099] Here, if no action control is set for the user avatar (No in S1001), the virtual object management unit 303 advances the process to S1003. In S1003, since no operation control is set for the user avatar, the virtual object management unit 303 operates the user avatar according to the user avatar operation information received by the operation information acquisition unit 307 in S508 above. Then, the process of this flowchart ends.

[0100] On the other hand, if action control is set for the user avatar (Yes in S1001), the virtual object management unit 303 advances the process to S1002. In S1002 , the virtual object management unit 303 acquires the action control information set for the user avatar from the virtual object control information management unit 306 . Next, in S1004, the virtual object management unit 303 determines whether the movement restricting shape of the movement control information acquired in S1002 matches or partially matches the user avatar operation information received by the operation information acquisition unit 307 in S508.

[0101] Here, if the movement restriction shape of the movement control information acquired in S1002 above matches or partially matches the user avatar operation information received by the operation information acquisition unit 307 in S508 above (Yes in S1004), the virtual object management unit 303 proceeds to S1005. In S1005, the virtual object management unit 303 does not allow any action that matches or partially matches the action limiting shape of the action control information acquired in S1002 among the user avatar operation information received by the operation information acquisition unit 307 in S508, and moves the user avatar according to the user avatar operation information for any action that does not match the action limiting shape. Then, the processing of this flowchart ends.

[0102] On the other hand, if the movement restricting shape of the movement control information acquired in S1002 above does not match the user avatar operation information received by the operation information acquisition unit 307 in S508 above and does not partially match (No in S1004), the virtual object management unit 303 proceeds to processing in S1006. In S1006, the virtual object management unit 303 moves the user avatar in accordance with the user avatar operation information, and then ends the processing of this flowchart. In addition, since the process of S518 in the third embodiment is similar to S510, the description thereof will be omitted.

[0103] Next, a case where input restrictions are imposed on the user as in the second embodiment will be described. Table 7 below shows an example of data of control information for each virtual object managed by the virtual object management unit 303 of the third embodiment when input restrictions are imposed on the user. [Table 7]

[0104] The user ID column is the same as that explained in Table 1, and indicates the user ID who sets the operation control. The input restriction range column indicates the shape for which input is restricted. In Table 7, as an example, the value is 3D model data in various formats, and if the user's body or part of the body matches or partially matches, the operation input will not be operated. In Table 7, the input restriction shape is specified by 3D model data as an example, but it may be specified by other means.

[0105] Figure 9B is a diagram showing an example of an input UI for controlling the behavior of a user avatar when input restrictions are imposed on a user in the third embodiment, and corresponds to an image from the user's perspective for userA, whose user ID is managed in the user management table of Table 1. Reference numeral 931 denotes the display 206 of the client terminal 121 used by userA. 941 shows a user's hand in the real world with the index finger pointed up. 942 shows the user's real-world arm in a fist pump position. 941 to 943 are multiple-choice input forms showing patterns for controlling actions, 941 and 942 are options for restricting input, and 943 is an option for releasing the control of actions. By selecting 941 or 942, the user can restrict the input of a specific shape. By selecting 943, the user can release the input restriction. The options for restricting input are not limited to the above, and any user actions or poses that can be detected by various sensors provided in the client terminal (or connectable to the interface 208) can be selected as options. In addition, the actions and poses of the user in the real world may be photographed by the camera 207 or the like, and the photographed data may be registered in the virtual space management system 111 as data for input restriction. In addition, the virtual space management system 111 may use artificial intelligence (AI) or the like to compare the data for input restriction registered by the user with the operation information of the user avatar to make a similarity judgment or the like, and execute control for restricting the actions of the user avatar.

[0106] Even when restricting input to a user, the overall processing sequence of the third embodiment is similar to that of the first embodiment, but the processing in which the virtual object management unit 303 operates the user avatar in S510 and S518 is different. Here, the specific processing content of S510 in the third embodiment will be described with reference to the flowchart in Fig. 10(b).

[0107] FIG. 10(b) is a flowchart showing an example of a user avatar operation process when input restrictions are imposed on the user in the third embodiment, and is executed by the software configuration of the virtual space management system 111 shown in FIG. In S1011, the virtual object management unit 303 acquires information about the user avatar that is the target of the operation of the user avatar operation information received by the operation information acquisition unit 307 in S508 of Figure 5 from the virtual object management unit 303, and determines whether input control is set for the user performing the operation.

[0108] Here, if input control has not been set for the user performing the operation (No in S1011), the virtual object management unit 303 advances the process to S1013. In S1013, since no input control is set for the user, the virtual object management unit 303 operates the user avatar in accordance with the user avatar operation information received by the operation information acquisition unit 307 in S508 above. Then, the process of this flowchart ends.

[0109] On the other hand, if input control is set for the user performing the operation (Yes in S1011), the virtual object management unit 303 advances the process to S1014. In S1014, the virtual object management unit 303 determines whether the user avatar operation information received by the operation information acquisition unit 307 in S508 above and the input restriction shape of the input control information acquired in S1012 above match or partially match.

[0110] Here, if the input restriction shape of the input control information acquired in S1012 above matches or partially matches the user avatar operation information received by the operation information acquisition unit 307 in S508 above (Yes in S1014), the virtual object management unit 303 proceeds to S1015. In S1015, the virtual object management unit 303 does not operate the user avatar according to operation information that matches or partially matches the input restriction shape of the input control information acquired in S1012 above among the user avatar operation information received by the operation information acquisition unit 307 in S508 above, but operates the user avatar according to the operation information that does not match. Then, the processing of this flowchart ends.

[0111] On the other hand, if the input restriction shape of the input control information acquired in S812 does not match and partially match the user avatar operation information received by the operation information acquisition unit 307 in S508 (No in S1014), the virtual object management unit 303 proceeds to S1016. In S1016, the virtual object management unit 303 moves the user avatar in accordance with the user avatar operation information, and then ends the processing of this flowchart. Incidentally, the process of S518 in the third embodiment is similar to the process of S510 described above, and therefore the description thereof will be omitted.

[0112] As described above, by setting a specific action restriction shape for a virtual object, or by setting a specific input restriction shape for a user, it is possible to control the action of a user avatar so that it does not perform a specific action. This method makes it possible to control the action of a shape that the user does not want to perform, and to communicate with other user avatars.

[0113] In addition, by combining the configurations shown in each of the above embodiments, the virtual space management system 111 that controls the movement of the avatar in the virtual space may be provided with an action restriction mode that restricts the movement of the virtual object (avatar) as shown in the first embodiment or the first half of the third embodiment, and an input restriction mode that restricts input for the user as shown in the second embodiment or the second half of the third embodiment, and the action restriction mode or the input restriction mode may be set for each user by the user, administrator, etc. Also, the enable / disable of the restriction mode may be set for each user, and when the restriction mode is set to be enabled, the action restriction mode or the input restriction mode may be set. Also, either the action restriction mode or the input restriction mode may be set as the initial setting. Note that these settings may be set collectively by the administrator for all users. The virtual space management system 111 executes control for restricting the movement of the avatar based on the detection data of the user corresponding to the avatar (such as user avatar operation information according to the detection of the user's movement) based on either the action restriction mode or the input restriction mode.

[0114] Also, to prevent an unexpected user action from causing the arm or the like of the user's user avatar to hit another user's user avatar, avatar restriction information that restricts the action of a user's user avatar on other avatars may be registered in the virtual space management system 111. Then, based on this avatar restriction information, the virtual space management system 111 executes control to restrict the action of the avatar based on the detection data of the user corresponding to the avatar.

[0115] According to each embodiment described above, it is possible to control the user avatar so that it does not move or so that only a part of the user avatar does not move when the user does not want it to. It is also possible to control the user avatar so that it does not perform actions that the user does not intend. In other words, it is possible to prevent the user avatar from performing actions that are not intended by the user, thereby making it possible to significantly improve usability.

[0116] It goes without saying that the configurations and contents of the various data described above are not limited to those described above, and the data may have various configurations and contents depending on the application and purpose. Although one embodiment has been described above, the present invention can be embodied, for example, as a system, an apparatus, a method, a program, a storage medium, etc. Specifically, the present invention may be applied to a system composed of multiple devices, or may be applied to an apparatus composed of a single device. Furthermore, any combination of the above-described embodiments is also included in the present invention.

[0117] Other embodiments The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. Furthermore, the present invention may be applied to a system made up of a plurality of devices, or to an apparatus made up of a single device. The present invention is not limited to the above-mentioned embodiment, and various modifications (including organic combinations of the embodiments) are possible based on the spirit of the present invention, and are not excluded from the scope of the present invention. In other words, the present invention includes all configurations that combine the above-mentioned embodiments and their modifications.

[0118] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) A system for controlling the movement of an avatar in a virtual space, comprising: A system characterized by having a control means that executes control to restrict the movement of an avatar based on detection data of a user corresponding to the avatar, based on either an operation restriction mode that restricts the movement of the avatar or an input restriction mode that restricts user input. (Configuration 2) a first setting means for setting a motion of the avatar to be restricted in the motion restriction mode; the first setting means sets a motion of the avatar to be restricted by designating a part of the avatar to be restricted; The system described in configuration 1, wherein the control means limits the movement of the set part of the avatar. (Configuration 3) a first setting means for setting a motion of the avatar to be restricted in the motion restriction mode; the first setting means sets a motion of the avatar to be restricted by designating a shape of the avatar to be restricted; The system described in configuration 1, wherein the control means restricts the movement of the avatar so as not to form a shape that matches or partially matches the set avatar shape. (Configuration 4) 4. The system according to configuration 2 or 3, wherein the first setting means sets the avatar movement to be restricted by selecting from a plurality of options. (Configuration 5) a second setting means for setting a user input to be restricted in the input restriction mode; the second setting means sets the input of the user to be restricted by designating a body part of the user to be restricted; The system according to any one of configurations 1 to 4, wherein the control means limits the movement of the avatar in accordance with the detection data of the set part of the user among the detection data of the user corresponding to the avatar. (Configuration 6) a second setting means for setting a user input to be restricted in the input restriction mode; The second setting means sets the input of a user to be restricted by designating a shape of the user to be restricted; The system described in any one of configurations 1 to 4, characterized in that the control means limits the movement of the avatar in accordance with detection data of a user corresponding to the avatar that matches or partially matches the set user shape within a range of detection data. (Configuration 7) 7. The system according to configuration 5 or 6, wherein the second setting means sets the input of the user to be restricted by selecting from a plurality of options. (Configuration 8) The system according to configuration 5 or 6, wherein the second setting means specifies the shape of the user to be restricted from input by using photographed data of the shape of the user to be restricted from input. (Method 1) A method for controlling a system that controls an avatar's movement in a virtual space, comprising: A system control method comprising a step of executing control for restricting the movement of an avatar based on detection data of a user corresponding to the avatar, based on either an operation restriction mode that restricts the movement of the avatar or an input restriction mode that restricts user input.

Claims

1. A system for controlling the movement of an avatar in a virtual space, comprising: A system characterized by having a control means that executes control to restrict the movement of an avatar based on detection data of a user corresponding to the avatar, based on either an operation restriction mode that restricts the movement of the avatar or an input restriction mode that restricts user input.

2. a first setting means for setting an avatar action to be restricted in the action restriction mode; the first setting means sets a motion of the avatar to be restricted by designating a part of the avatar to be restricted; The system according to claim 1 , wherein the control means limits the movement of the set part of the avatar.

3. a first setting means for setting an avatar action to be restricted in the action restriction mode; the first setting means sets a motion of the avatar to be restricted by designating a shape of the avatar to be restricted; 2. The system according to claim 1, wherein said control means restricts the movement of the avatar so as not to form a shape that coincides or partially coincides with the set avatar shape.

4. 4. The system according to claim 2, wherein the first setting means sets the avatar's movement to be restricted by selecting from a plurality of options.

5. a second setting means for setting a user's input to be restricted in the input restriction mode; the second setting means sets the input of the user to be restricted by designating a body part of the user to be restricted; 2. The system according to claim 1, wherein the control means restricts the movement of the avatar in accordance with the detection data of the set part of the user among the detection data of the user corresponding to the avatar.

6. a second setting means for setting a user's input to be restricted in the input restriction mode; The second setting means sets the input of a user to be restricted by designating a shape of the user to be restricted; The system described in claim 1, characterized in that the control means limits the movement of the avatar in accordance with detection data of a user corresponding to the avatar that matches or partially matches the set user shape within a range of detection data.

7. 7. The system according to claim 5, wherein the second setting means sets the input of the user to be restricted by selecting from a plurality of options.

8. 7. The system according to claim 5, wherein the second setting means specifies the shape of the user to be restricted from input by using photographed data of the shape of the user to be restricted from input.

9. A method for controlling a system that controls the movement of an avatar in a virtual space, comprising: A system control method comprising a step of executing control for restricting the movement of an avatar based on detection data of a user corresponding to the avatar, based on either an operation restriction mode that restricts the movement of the avatar or an input restriction mode that restricts user input.

Images