Programs and Information Processing Systems

Abstract

To improve user experience. [Solution] The program causes the computer to function as a control means for operating operation objects and user objects placed in a virtual space based on user operations, and an image processing means for displaying a virtual space image, which is an image of the virtual space based on a virtual camera placed in the virtual space. The control means moves the user object in accordance with the operation of the operation object when an operation is performed in a first direction on the operation object, or when an operation is performed in a second direction after an operation in the first direction. If no operation is performed on the operation object in the first direction, but an operation is performed in the second direction, the user object is not moved, and the orientation of the virtual camera is changed in accordance with the operation of the operation object.

Description

Technical Field

[0001] The present disclosure relates to programs and information processing systems.

Background Art

[0002] There is known a technique for moving a user object in a virtual space based on a user's operation input and generating a virtual space image based on a virtual camera that moves according to the movement of the user object. Patent Document 1 discloses a technique for detecting the display of a user's face and controlling the facial expression of an avatar in a virtual space according to the detected display of the user's face.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present disclosure aims to improve the operability of a user.

Means for Solving the Problems

[0005] A program according to an embodiment shown in the present disclosure causes a computer to function as control means for operating an operation object and a user object arranged in a virtual space based on an operation by a user, and image processing means for displaying a virtual space image that is an image of the virtual space based on a virtual camera arranged in the virtual space, wherein the control means moves the user object according to the operation of the operation object when an operation in a first direction is performed on the operation object, or when an operation in a second direction is performed after the operation in the first direction. If the operation object is not operated in the first direction but is operated in the second direction, the user object is not moved, and the orientation of the virtual camera is changed according to the operation of the operation object.

[0006] The information processing system according to one embodiment shown in this disclosure is: An information processing system comprising one or more information processing devices, A control means for operating operation objects and user objects placed in a virtual space based on user operations, The system includes an image processing means for displaying a virtual space image, which is an image of the virtual space based on a virtual camera placed in the virtual space. The control means is When an operation is performed on the operation object in a first direction, or when an operation is performed in a second direction after an operation in the first direction, the user object is moved according to the movement of the operation object. If the operation object is not operated in the first direction but is operated in the second direction, the user object is not moved, and the orientation of the virtual camera is changed according to the operation of the operation object. [Effects of the Invention]

[0007] According to this disclosure, it is possible to improve user experience. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a diagram showing the configuration of an information processing system according to an embodiment of this disclosure. [Figure 2] Figure 2 is a block diagram showing the configuration of the terminal device shown in Figure 1. [Figure 3] Figure 3 shows an example of a virtual space image displayed on the display unit shown in Figure 2. [Figure 4] Figure 4 is a diagram illustrating the movement operation of the user object shown in Figure 3. [Figure 5] FIG. 5 is a diagram showing an example of a virtual space image indicating a state where the orientation of the virtual camera has been changed. [Figure 6] FIG. 6 is a flowchart showing an example of the operation flow of the program according to the first embodiment of the present disclosure. [Figure 7] FIG. 7 is a diagram (part 1) showing an example of a virtual space image for explaining the warp movement of the user object. [Figure 8] FIG. 8 is a diagram (part 2) showing an example of a virtual space image for explaining the warp movement of the user object. [Figure 9] FIG. 9 is a diagram showing an example of a state where the position of the position specifying object shown in FIG. 7 has been changed. [Figure 10] FIG. 10 is a diagram showing an example of a state where another object is arranged at the position of the position specifying object. [Figure 11] FIG. 11 is a flowchart showing an example of the operation flow of the program according to the second embodiment of the present disclosure. [Figure 12] FIG. 12 is a diagram (part 1) showing an example of a virtual space image for explaining the camera position control mode by the object control unit shown in FIG. 2. [Figure 13] FIG. 13 is a diagram (part 2) showing an example of a virtual space image for explaining the camera position control mode by the object control unit shown in FIG. 2. <000​​​​​​​​​​​​Hereinafter, embodiments of this technical idea will be described in detail with reference to the drawings. In the following description, the same components etc. are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated.

[0010] <First Embodiment> [Configuration of Information Processing System] Hereinafter, an information processing system 1 that moves a user object in a virtual space based on a user's operation input and provides a user with a virtual space image generated based on a virtual camera arranged in the virtual space will be described.

[0011] FIG. 1 is a diagram showing the configuration of an information processing system 1 according to an embodiment of the present disclosure. As shown in FIG. 1, the information processing system 1 includes one or more terminal devices 10 and a server 20, and these devices are communicably connected to each other by a network 13. In FIG. 1, terminal devices 10A and 10B used by a plurality of users are shown.

[0012] The terminal device 10 is connected to the network 13 by communicating with a wireless router 14 installed in a facility such as a house. Note that the terminal device 10 may be configured to be connected to the network 13 by wired communication.

[0013] The terminal device 10 is a computer (information processing device) used by a user. The terminal device 10 is, for example, a head-mounted device (HMD) including VR (Virtual Reality) glasses, AR (Augmented Reality) glasses, or MR (Mixed Reality) glasses. Note that the terminal device 10 may be a contact-type device that can be worn on the eyes like contact lenses.

[0014] Terminal device 10 executes an application program installed via a platform that distributes apps, etc. Alternatively, terminal device 10 may execute a program obtained via website browsing software, i.e., a web browser, instead of an application program. By executing the program, terminal device 10 generates a virtual space image and displays the virtual space image. In generating the virtual space image, terminal device 10 sends and receives various data with server 20 as needed.

[0015] The virtual space is generated using XR technologies such as VR, AR, MR, or SR (Substitutional Reality). The virtual space may be common to multiple users or different for each user. That is, multiple user objects may exist in one virtual space, or one user object may exist in one virtual space.

[0016] Furthermore, the virtual space is not limited to one generated using XR technology. For example, it may be a virtual space generated on a web browser. It may also be a virtual space in which XR technology is incorporated into a part of a virtual space generated on a web browser. It may also be a system that transitions from a virtual space reproduced on a web browser to a virtual space generated using XR technology.

[0017] Server 20, for example, receives information regarding user input from terminal device 10 and, in accordance with the received information, transmits virtual space information necessary for generating the virtual space to terminal device 10. The virtual space information includes information for generating various virtual objects such as virtual cameras and user objects that are placed in the virtual space.

[0018] [Terminal device configuration] (Configuration overview) Figure 2 is a block diagram showing the configuration of the terminal device 10 shown in Figure 1. As shown in Figure 2, the terminal device 10 comprises a communication unit 21, a control unit 22, a storage unit 23, a memory 24, a display unit 25, and an operation detection unit 26.

[0019] The communication unit 21 functions as an interface for the terminal device 10 to communicate with an external device such as the server 20 (see Figure 1). For example, when virtual space information is transmitted from the server 20 to the terminal device 10, the communication unit 21 receives the virtual space information and outputs it to the control unit 22.

[0020] The storage unit 23 includes a storage device such as an HDD (Hard Disk Drive) or flash memory. Specifically, the storage unit 23 stores programs 41 and user information 42, etc. Program 41 is a program for the terminal device 10 to execute various processes. User information 42 includes information about the user, such as identification information for the terminal device 10 and identification information for each user.

[0021] Memory 24 includes a storage device such as ROM (Read Only Memory) or RAM (Random Access Memory). Various data generated in connection with the operation of the control unit 22, such as information about user objects operated by the user, are temporarily stored in Memory 24.

[0022] The display unit 25 is a monitor on which virtual space images are displayed. For example, if the terminal device 10 is an HMD (Head-Mounted Display), the display unit 25 is positioned within the user's field of view when the user is wearing the terminal device 10.

[0023] The operation detection unit 26 has a hand tracking function. More specifically, the operation detection unit 26 has, for example, an imaging unit that periodically captures images of the area in front of the terminal device 10. When the operation detection unit 26 detects the movement of the user's hand or arm in front of the terminal device 10 based on the captured images, it outputs operation information indicating the detection result to the control unit 22.

[0024] Specifically, the operation detection unit 26 performs image analysis processing on the captured image to determine whether or not a user's hand is present in the captured image. If a user's hand is present in the captured image, the operation detection unit 26 detects the coordinates of the tip of the user's index finger and the tip of the thumb in the captured image and outputs operation information indicating these coordinates to the control unit 22.

[0025] The operation detection unit 26 is not limited to detecting the coordinates of the tip of the index finger and the tip of the thumb, but may also detect the coordinates of the tips of other fingers. However, the tips of the index finger and the thumb are suitable targets for detection because they are located close to each other and have a large difference in length.

[0026] The control unit 22 is a processor, for example, including a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a GPU (Graphics Processing Unit). By operating according to the program 41, the control unit 22 performs the functions of a space generation unit 31, a placement unit 32, an object control unit 33, an image processing unit 34, and a chat processing unit 35.

[0027] The space generation unit 31 generates a virtual space and virtual objects such as virtual cameras and user objects placed in the virtual space based on virtual space information transmitted from the server 20 (see Figure 1). The virtual objects generated by the space generation unit 31 include other user objects that are operated by other users.

[0028] The placement unit 32 places various virtual objects, such as a virtual camera, user objects, and other user objects, into the virtual space. The image processing unit 34 generates a virtual space image, which is an image captured from the virtual camera within the virtual space, and displays the generated virtual space image on the display unit 25.

[0029] Figure 3 shows an example of a virtual space image displayed on the display unit 25 shown in Figure 2. As shown in Figure 3, for example, a virtual camera is positioned behind a user object 601 operated by the user, and the virtual space image, which is an image captured by the virtual camera, is displayed on the display unit 25.

[0030] (Move user objects) Referring to Figures 2 and 3, the placement unit 32 further places an operation panel 602 in the virtual space. For example, the operation panel 602 is positioned in front of the virtual camera's field of view and at a predetermined distance from the virtual camera. The operation panel 602 is equipped with menu buttons 603 for making various settings, and an operation object 604 used for moving the user object 601. The operation object 604 is, for example, stick-shaped and can be tilted in the forward / backward and left / right directions.

[0031] Furthermore, when the placement unit 32 receives operation information from the operation detection unit 26, it places a hand object 605, which is an object resembling a hand, in the virtual space in front of the field of view of the virtual camera.

[0032] The object control unit 33 operates the hand object 605 based on user operations. More specifically, the object control unit 33 moves the index finger and thumb of the hand object 605 based on multiple operation information from the operation detection unit 26. As a result, the index finger and thumb of the hand object 605 move in accordance with the actual movements of the user's index finger and thumb.

[0033] Furthermore, the object control unit 33 enables the operation on the operation object 604 when a predetermined action is performed by the user.

[0034] More specifically, for example, when the object control unit 33 receives operation information from the operation detection unit 26, it determines, based on the operation information, whether or not a first operation has been performed by the user while the user's index finger and thumb are included in the first region of the captured image. The first operation is an operation to bring the index finger and thumb closer together (hereinafter referred to as a "grasping operation"). If the object control unit 33 determines that the user has performed the operation, it enables the operation on the operation object 604.

[0035] Furthermore, after enabling an operation on the operation object 604, the object control unit 33 disables the operation on the operation object 604 if a predetermined action is performed by the user.

[0036] More specifically, for example, when the object control unit 33 receives new operation information from the operation detection unit 26, it determines, based on the operation information, whether a second operation has been performed by the user while the user's index finger and thumb are included in the first region of the captured image. The second operation is an operation to move the index finger and thumb away from each other (hereinafter referred to as the "separation operation"). If the object control unit 33 determines that the user has performed the operation, it switches the operation on the operation object 604 from enabled to disabled.

[0037] Furthermore, the first action is not limited to grasping using the thumb and index finger, but may also include pushing, gripping, bending the wrist or elbow, etc. Also, the second action is not limited to releasing the thumb and index finger, but may also include pulling, spreading the hand, straightening a bent wrist or elbow, etc.

[0038] Furthermore, the object control unit 33 moves the user object 601 in response to user operations during the period in which an operation on the operation object 604 is active.

[0039] Figure 4 is a diagram illustrating the movement operation of the user object 601 shown in Figure 3. For example, suppose the user moves their hand forward during the period when the operation on the operation object 604 is active. In this case, the object control unit 33 detects that the user has moved their hand forward based on the movement information from the operation detection unit 26 and tilts the operation object 604 forward. The object control unit 33 also moves the user object 601 forward in response to the movement of the operation object 604 and causes the virtual camera to follow the user object 601.

[0040] Furthermore, for example, suppose the user moves their hand backward during the period when an operation on the operation object 604 is active. In this case, the object control unit 33 detects that the user has moved their hand backward based on the operation detection unit 26 and tilts the operation object 604 backward. The object control unit 33 also moves the user object 601 backward in response to the movement of the operation object 604 and causes the virtual camera to follow the user object 601.

[0041] User object 601 can move not only forward and backward, but also left and right. For example, if the user moves their hand forward or backward and then to the right while an operation on operation object 604 is active, user object 601 will move to the right.

[0042] Furthermore, for example, if the user moves their hand forward or backward and then to the left while an operation on the operation object 604 is active, the user object 601 will move to the left. The example shown in Figure 4 illustrates the state in which the user object 601 is moving to the left.

[0043] (Changing the orientation of the virtual camera) The object control unit 33 can change the orientation of the virtual camera in response to user operations on the operation object 604. Figure 5 shows an example of a virtual space image showing the state in which the orientation of the virtual camera has been changed.

[0044] For example, suppose that during the period when an operation on the operation object 604 is active, the user does not perform an operation on the operation object 604 in the first direction, but instead performs an operation in the second direction. In this case, the object control unit 33 does not operate the user object 601, but changes the orientation of the virtual camera. In this embodiment, an example of the first direction is the front-back direction, and an example of the second direction is the left-right direction.

[0045] Furthermore, the operation in the first direction, i.e., the forward / backward direction, refers to at least one of the operations in the forward direction and the backward direction. Also, the operation in the second direction, i.e., the left / right direction, refers to at least one of the operations in the left direction and the right direction.

[0046] Specifically, suppose that during the period when an operation on the operation object 604 is active, the user moves their hand to the left without moving it forward or backward. In this case, as shown in Figure 5, the object control unit 33 changes the orientation of the virtual camera to the left by rotating the virtual camera to the left while keeping the position of the user object 601 fixed. Alternatively, if the object control unit 33 changes the orientation of the virtual camera to the left, it may control the user object 601 to align its orientation with the orientation of the virtual camera by rotating the user object 601 to the left.

[0047] Furthermore, for example, suppose that during the period when an operation on the operation object 604 is active, the user moves their hand to the right without moving it forward or backward. In this case, the object control unit 33 changes the orientation of the virtual camera to the right by rotating the virtual camera to the right while keeping the position of the user object 601 fixed. Alternatively, if the object control unit 33 changes the orientation of the virtual camera to the right, it may also control the user object 601 to align its orientation with the orientation of the virtual camera by rotating the user object 601 to the right.

[0048] Furthermore, the object control unit 33 may change the amount of change in the orientation of the virtual camera according to the amount of movement of the user's hand in the left-right direction. For example, the object control unit 33 increases the amount of change in the orientation of the virtual camera the greater the amount of movement of the user's hand in the left-right direction.

[0049] [Operation Flow] Figure 6 is a flowchart showing an example of the operation flow of a program according to the first embodiment of this disclosure. The order of the processes constituting each flowchart described herein is not limited to the extent that no inconsistencies or contradictions occur in the processing content, and they may be executed in parallel. In addition, some of the processes constituting each flowchart described herein may be omitted.

[0050] Referring to Figures 2 and 6, first assume that the virtual space image generated by the image processing unit 34 is displayed on the display unit 25 (step S11). In this situation, if the object control unit 33 has not received operation information from the operation detection unit 26 (NO in step S12), it waits until it receives operation information.

[0051] On the other hand, if the object control unit 33 receives operation information from the operation detection unit 26 (YES in step S12), it places the hand object 605 in the virtual space and controls the hand object 605 to operate based on the new operation information from the operation detection unit 26 (step S13).

[0052] Next, the object control unit 33 determines, based on the operation information from the operation detection unit 26, whether or not the first operation has been performed by the user while the user's index finger and thumb are included in the first region of the captured image (step S14). If the object control unit 33 determines that the user has not performed the operation (NO in step S14), it proceeds to step S12 and waits until it receives new operation information.

[0053] Furthermore, in step S12, if a predetermined time has elapsed since the previous operation information was received and the hand object 605 is located in the virtual space, the object control unit 33 deletes the hand object 605 from the virtual space.

[0054] Furthermore, if the object control unit 33 determines in step S14 that the above action has been performed by the user (YES in step S14), it enables the operation on the operation object 604 (step S17).

[0055] Next, the object control unit 33 determines whether or not an operation has been performed on the operation object 604 in the forward or backward direction based on the operation information from the operation detection unit 26 (step S18). If an operation has been performed on the operation object 604 in the forward or backward direction (YES in step S18), the object control unit 33 moves the user object 601 in accordance with the operation of the operation object 604 (step S19).

[0056] Next, the object control unit 33 determines, based on the operation information from the operation detection unit 26, whether or not the user has performed the second operation while the user's index finger and thumb are included in the first region of the captured image (step S20). If the object control unit 33 determines that the user has not performed the above operation (NO in step S20), it repeats the operations from step S18 onward.

[0057] On the other hand, if the object control unit 33 determines that the above action has been performed by the user (YES in step S20), it disables the operation on the operation object 604 (step S21). Then, the object control unit 33 proceeds to step S12 and waits until it receives new operation information.

[0058] Furthermore, in step S18, it is assumed that no operation has been performed on the operation object 604 in the forward or backward direction ("NO" in step S18). In this case, the object control unit 33 determines whether or not an operation has been performed on the operation object 604 in the left or right direction based on the operation information from the operation detection unit 26 (step S22).

[0059] Then, if an operation is performed on the operation object 604 in the left or right direction (YES in step S22), the object control unit 33 changes the orientation of the virtual camera according to the operation of the operation object 604 (step S23). Then, the object control unit 33 proceeds to step S12 and waits until it receives new operation information.

[0060] Furthermore, in step S22, if no operation is performed on the operation object 604 in the left or right direction (NO in step S22), the object control unit 33 does not operate either the user object 601 or the virtual camera, proceeds to step S12, and waits until it receives new operation information.

[0061] As described above, in the first embodiment, the target of operation for moving the user object 601 and the target of operation for changing the orientation of the virtual camera are both the operation object 604. That is, the user can move the user object 601 or change the orientation of the virtual camera by operating on the operation object 604. This improves user operability.

[0062] Furthermore, since the target of the operation is the controllable object 604 placed in the virtual space, there is no need to set up hardware such as a controller, making it easy and convenient for even unfamiliar users to play.

[0063] <Second Embodiment> Next, a second embodiment of the present disclosure will be described. In the first embodiment described above, a configuration was described in which the orientation of the virtual camera is changed in response to user operations on the operation object 604. In contrast, the second embodiment will describe a configuration in which the user object 601 is moved in a specific manner that is faster than normal movement in response to user movement.

[0064] Herein, "normal movement" refers to the movement of the user object 601 accompanying the tilting of the operation object 604. Hereafter, a specific mode of movement that is faster than the normal movement of the user object 601 will be referred to as "warp movement." Warp movement may be instantaneous movement of the user object 601, or it may be movement at a speed in which the movement path of the user object 601 is recognizable.

[0065] [Warping user objects] (Overview of warp travel) Figures 7 and 8 show examples of virtual space images to illustrate the warp movement of a user object. As described in the first embodiment, the object control unit 33 (see Figure 2) enables the operation on the operation object 604 when the user performs a first action while the user's index finger and thumb are included in the first region of the captured image. The object control unit 33 also disables the operation on the operation object 604 when the user performs a second action while the user's index finger and thumb are included in the first region of the captured image.

[0066] This first region corresponds to the virtual region R1 in the virtual space image. That is, if the tips of the user's index finger and thumb are included in the first region of the captured image, the hand object 605 will be displayed inside the virtual region R1 in the virtual space image, as shown in Figure 3.

[0067] Furthermore, if the tips of the user's index finger and thumb are included in the second region, which is outside the first region of the captured image, the hand object 605 will be displayed outside the virtual region R1 of the virtual space image, as shown in Figure 7.

[0068] In the second embodiment, when the object control unit 33 receives operation information from the operation detection unit 26, it determines, based on the operation information, whether or not the first operation was performed by the user while the user's hand was included in the second region of the captured image.

[0069] The first action here is, for example, the same action as the first action described in the first embodiment, such as a grasping action by bringing the index finger and thumb closer together. Hereinafter, an operation performed by the user through the first action while the user's hand is included in the second region of the captured image will be referred to as the "warp position display operation" (first operation).

[0070] Furthermore, the object control unit 33 determines, based on the operation information from the operation detection unit 26, whether or not a second operation was performed by the user while the user's hand was included in the second region of the captured image.

[0071] The second action here is, for example, the same action as the second action described in the first embodiment, such as separating the index finger and thumb. Hereinafter, an operation performed by the user through the second action while the user's hand is included in the second region of the captured image will be referred to as a "warp execution operation".

[0072] Furthermore, the first operation in the second embodiment may be the same as the second operation described in the first embodiment, and the second operation in the second embodiment may be the same as the first operation described in the first embodiment.

[0073] When the object control unit 33 detects a warp position display operation by the user, it places a position designation object 701 that specifies the destination of the user object 601, as shown in Figure 7, for example. The object control unit 33 also places a parabola 702 connecting the position designation object 701 and the hand object 605 in the virtual space.

[0074] Then, when the object control unit 33 detects a warp execution operation by the user in this state, it moves the user object 601 to the position of the position-specified object 701 by warp movement, as shown in Figure 8. At this time, the object control unit 33 makes the virtual camera follow the user object 601.

[0075] It should be noted that the first operation described in the first embodiment, i.e., the first operation when enabling an operation on the operation object 604, and the first operation in the second embodiment, i.e., the first operation performed when the warp position display operation is performed, may be different operations.

[0076] Furthermore, the second operation described in the first embodiment, namely the second operation when disabling an operation on the operation object 604, and the second operation in the second embodiment, namely the second operation performed during a warp execution operation, may be different operations.

[0077] (Change of destination) The user can change the position, i.e., the destination, of the designated object 701 by performing a predetermined operation (the second operation) while continuing the warp position display operation. As described above, the warp position display operation is an operation based on the movement of the fingers of the hand, such as the index finger and thumb (the first part), whereas the predetermined operation for changing the position of the designated object 701 is an operation based on the movement of a part of the body such as the elbow or wrist (the second part).

[0078] Figure 9 shows an example of a state in which the position of the position-designated object 701 shown in Figure 7 has been changed. For example, as shown in Figure 9, suppose the user bends their elbow or wrist while continuing the warp position display operation, bringing their fingers closer to themselves. In this case, the object control unit 33 moves the position-designated object 701 toward the user, that is, toward the virtual camera.

[0079] Furthermore, suppose the user, while continuing to operate the warp position display, extends their elbow or bends their wrist, thereby moving their fingers away from themselves. In this case, the object control unit 33 moves the position-specified object 701 in the depth direction, that is, away from the virtual camera.

[0080] Furthermore, the object control unit 33 may be configured to move the position-designated object 701 in the depth direction when the user moves their fingers toward themselves, and to move the position-designated object 701 toward the user when the user moves their fingers toward themselves.

[0081] Furthermore, the object control unit 33 can move the position of the position-specified object 701 in response not only to changes in the depth direction of the user's fingers, but also to changes in the left-right direction, the height direction, and the orientation.

[0082] For example, if the user moves their finger upward while continuing the warp position display operation, the object control unit 33 moves the position-specified object 701 in the depth direction. Also, if the user moves their finger downward while continuing the warp position display operation, the object control unit 33 moves the position-specified object 701 towards the user.

[0083] (Processing when a finger included in the captured image moves from the second region to the first region) Assuming that the user moves their hand while continuing to operate the warp position display, the index finger and thumb included in the captured image move from the second region to the first region, the object control unit 33 determines that the first action has been performed by the user while the user's hand is included in the first region of the captured image, and enables the operation on the operation object 604.

[0084] Then, the object control unit 33 turns off the display of the position-specified object 701 and the parabola 702, and performs operations such as moving the user object 601 or changing the orientation of the virtual camera, as described in the first embodiment.

[0085] Furthermore, if the user moves their hand while continuing to operate the warp position display, and the index finger and thumb included in the captured image move from the second region to the first region, the display of the position designation object 701 and the parabola 702 may be maintained. In this case, the position of the position designation object 701 is changed according to the movement of the user's hand.

[0086] (Handling cases where warp movement to a specified object is not possible) (a) When another object is located at the position of the specified object The object control unit 33 notifies the user that warp movement cannot be performed if other objects, such as other user objects or building objects, are located at the position of the position-specified object 701. For example, the object control unit 33 displays at least one of the position-specified object 701 and the parabola 702 in a display mode different from when warp movement is possible.

[0087] Figure 10 shows an example of a state where other objects are placed at the position of the designated object 701. For example, if warp movement is possible, the object control unit 33 displays the parabola 702 in green (solid line in Figure 9), as shown in Figure 9.

[0088] On the other hand, suppose another object (a tree object in the example shown in Figure 10) is placed at the location of the position-specified object 701, and warp movement cannot be performed. In this case, the object control unit 33 displays the parabola 702 in red (a dashed line in Figure 10), as shown in Figure 10. If warp movement cannot be performed, the position-specified object 701 may be marked with an "X" or the parabola 702 may blink, or other similar indications may be used.

[0089] Furthermore, the object control unit 33 may be configured to automatically change the position of the position-designated object 701 if another object is placed at the position of the position-designated object 701. In such a configuration, the object control unit 33 may, for example, move the position-designated object 701 to the front or left or right of the other object.

[0090] (b) When another object is located between the current position of the user object and the positioning object. Assume that another object is located between the current position of user object 601 and position-specifying object 701. In such a case, the object control unit 33 notifies the user that warp movement cannot be performed, for example, as if another object were located at the position of position-specifying object 701.

[0091] Furthermore, the object control unit 33 may be configured to enable warp movement even when other objects are positioned between the current position of the user object 601 and the position-specifying object 701. For example, the object control unit 33 may move the user object 601 so that it jumps over or passes through the other object during warp movement.

[0092] [Operation Flow] Figure 11 is a flowchart showing an example of the program's operation flow according to the second embodiment of this disclosure. The same aspects as the first embodiment, namely the movement of the user object 601 and the change in the orientation of the virtual camera, will not be repeated in this description. Furthermore, it is assumed that the acquisition of operation information from the operation detection unit 26 and the control of the hand object 605 are performed continuously.

[0093] Referring to Figures 2 and 11, first, the object control unit 33 determines whether or not a warp position display operation has been performed by the user based on the operation information from the operation detection unit 26 (step S31). If the object control unit 33 determines that no warp position display operation has been performed by the user (NO in step S31), it repeats the operation in step S31 each time it acquires operation information.

[0094] On the other hand, if the object control unit 33 determines that the user has performed a warp position display operation (YES in step S31), it places the position-specified object 701 and the parabola 702 in the virtual space (step S32).

[0095] Next, the object control unit 33 changes the position of the position-designated object 701 in response to a change in at least one of the user's hand's position in the depth direction, left-right direction, height direction, and orientation (step S33). If the user's hand's position or orientation does not change, the position of the position-designated object 701 is not changed.

[0096] Next, the object control unit 33 determines whether or not warp movement is possible (step S34). The object control unit 33 then determines that another object is located at the position of the position-specified object 701 and therefore warp movement of the user object 601 cannot be performed (NO in step S34).

[0097] In this case, the object control unit 33 changes the display mode of at least one of the position-specified object 701 and the parabola 702 to a display mode different from the one in which warp movement is possible (step S35). Then, the object control unit 33 proceeds to step S33.

[0098] On the other hand, the object control unit 33 determines in step S34 that it is possible to perform a warp movement on the user object 601 (YES in step S34). In this case, the object control unit 33 determines whether or not a warp operation has been performed by the user based on the new operation information from the operation detection unit 26 (step S36). If a warp operation has not been performed by the user (NO in step S36), the object control unit 33 performs the operations from step S33 onward again.

[0099] On the other hand, if the user performs a warp operation (YES in step S36), the object control unit 33 moves the user object 601 to the position of the designated object 701 by warp movement. At this time, the object control unit 33 also moves the virtual camera so that it follows the user object 601 (step S37). Then, the object control unit 33 repeats the operations from step S31 onwards.

[0100] As described above, in the second embodiment, the user object 601 can be moved to its destination in a short time by user operation. This improves user operability.

[0101] Furthermore, the user can distinguish between normal movement and warp movement of the user object 601 by changing the area in which the movement takes place. Therefore, there is no need for hardware such as controllers, and even inexperienced users can play easily and casually. In addition, because the movement time of the user object 601 is shortened, motion sickness caused by the virtual camera following the user object 601 can be reduced.

[0102] Furthermore, when user object 601 warps to location-specified object 701, the virtual camera moves to the vicinity of user object 601's destination, as described above. However, if the virtual camera movement occurs instantaneously, the viewpoint of the virtual space image changes drastically in a short time, making it difficult for the user to grasp where user object 601 has moved to.

[0103] Therefore, when user object 601 warps to location-specified object 701, the virtual camera may be configured to move to the vicinity of user object 601 at a speed sufficient to recognize the movement path of user object 601.

[0104] Furthermore, the object control unit 33 is not limited to a configuration in which both the position-specifying object 701 and the parabola 702 are placed in the virtual space when a warp position display operation is performed by the user. For example, the object control unit 33 may be configured to place the position-specifying object 701 but not the parabola 702. However, when the parabola 702 is placed, the movement path of the user object 601 becomes clear. For this reason, a configuration in which both the position-specifying object 701 and the parabola 702 are placed is more preferable.

[0105] <Third Embodiment> Next, a third embodiment of the present disclosure will be described. In the second embodiment described above, a configuration was described in which the user object 601 is warped in response to user operations. In contrast, the third embodiment will describe a configuration in which the movement of the virtual camera is controlled in response to user operations so that a virtual space image different from the normal one is displayed.

[0106] [Mode switching] Referring again to Figure 2, the object control unit 33 moves the user object 601 in the virtual space in response to user operations, as described above. At this time, the object control unit 33 controls the movement of the user object 601 and the virtual camera so that the virtual camera follows the user object 601. Hereinafter, this mode of operation will be referred to as the "normal mode" (first mode).

[0107] In the third embodiment, the object control unit 33 can control the movement of the user object 601 and the virtual camera so that the virtual camera does not follow the user object 601 when the user object 601 is moved. Hereinafter, this operating mode will be referred to as the "camera position control mode" (second mode). The object control unit 33 can switch between the normal mode and the camera position control mode in response to user operations.

[0108] Figures 12 and 13 are examples of virtual space images illustrating the camera position control mode by the object control unit 33 shown in Figure 2. For example, the operation panel 602 is provided with a toggle button 606 for switching between normal mode and camera position control mode. For example, if both of the user's hands are included in the first region of the captured image, the operation detection unit 26 identifies the coordinates of the tip of the index finger and the tip of the thumb for each of the right and left hands, and outputs operation information indicating these coordinates to the control unit 22.

[0109] When the object control unit 33 in the control unit 22 receives operation information output from the operation detection unit 26, it places a hand object 605A, which is modeled after the right hand, and a hand object 605B, which is modeled after the left hand, in the virtual area R1, as shown in Figure 12, for example. Then, based on the operation information from the operation detection unit 26, the object control unit 33 makes the hand object 605A move in the same way as the user's right hand, and the hand object 605B move in the same way as the user's left hand.

[0110] Furthermore, the object control unit 33 detects a user's switching operation based on operation information from the operation detection unit 26. The switching operation is performed, for example, by the user moving both hands so that one hand object 605 is positioned around the switching button 606 and the other hand object 605 grasps the operation object 604. When the object control unit 33 detects the user's switching operation, it switches between normal mode and camera position control mode.

[0111] In camera position control mode, the object control unit 33 fixes, for example, the position and orientation of the virtual camera and moves the user object 601 in accordance with the operation of the operation object 604. As a result, the user object 601 gradually moves away from the virtual camera, and in the virtual space image, for example as shown in Figure 12, the user object 601 is displayed as gradually smaller.

[0112] Furthermore, if the object control unit 33 detects a user switching operation while in camera position control mode, it terminates camera position control mode and switches to normal mode. That is, as shown in Figure 13, the object control unit 33 moves the virtual camera around the user object 601 after it has moved in camera position control mode. The object control unit 33 then controls the position of the virtual camera so that it follows the user object 601 in accordance with the movement of the user object 601.

[0113] [Details on moving user objects in camera position control mode] In camera position control mode, the movable range of the user object 601 is limited, for example, to the range included in the virtual space image.

[0114] Furthermore, similar to the warp movement of the user object 601 in the second embodiment, the movement of the user object 601 to locations where other objects, such as other user objects or building objects, are located is restricted. For example, if the object control unit 33 finds that another object is located in the movement path or destination of the user object 601, it moves the user object 601 to a location in front of that other object.

[0115] Furthermore, the object control unit 33 may control the user object 601 during movement in camera position control mode so that it jumps over, avoids, or passes through other objects placed in its movement path.

[0116] Furthermore, on the terminal device 10 of the user operating the user object 601, the user object 601 may appear to be moving as if passing through other objects, while on the terminal device 10 of other users, the user object 601 may appear to be moving as if jumping over or avoiding other objects.

[0117] [Voice chat function] Referring again to Figure 2, the chat processing unit 35 in the control unit 22 provides voice chat functionality to each user of multiple user objects located in the virtual space. Furthermore, the chat processing unit 35 can disable the voice chat function when the user object 601 is being moved in camera position control mode.

[0118] Figure 14 shows an example of a virtual space image displayed on another user's terminal device 10 when the voice chat function is disabled by the chat processing unit 35 shown in Figure 2. Here, user object 601A and user object 601B are located close to each other in the virtual space, and user object 601A is moving in camera position control mode. In this case, the chat processing unit 35 disables the voice chat function on terminal device 10A, which is the terminal device 10 of the user operating user object 601A.

[0119] Furthermore, the chat processing unit 35 in terminal device 10A sends movement information to server 20 via network 13, indicating that the user object 601A is moving in camera position control mode. In this case, the virtual space information sent from server 20 to terminal device 10B includes the above movement information.

[0120] The movement information transmitted from the server 20 to the terminal device 10B includes, for example, identification information of the terminal device 10A, identification information of the user object 601A, and position information indicating the position of the user object 601A before movement in camera position control mode.

[0121] Then, in the virtual space image displayed on the display unit 25 of the terminal device 10B, the user object 601A is displayed at the position it was in before moving in camera position control mode, as shown in Figure 14, for example. In addition, the virtual space image may display a message indicating, for example, that the voice chat function is stopped because the user object 601A is moving in camera position control mode.

[0122] Furthermore, the virtual space image displayed on the display unit 25 of the terminal device 10B may be displayed in a manner different from the normal mode, such as when the user object 601A moving in camera position control mode is displayed semi-transparently.

[0123] Furthermore, not only in terminal device 10B but also in terminal device 10A, a semi-transparent user object 601A may be displayed at the position before the camera position control mode started, indicating that the voice chat function is disabled.

[0124] Furthermore, if the user object 601A is displayed on the display unit 25 of the terminal device 10B at the position before the camera position control mode started, the voice chat function may be continuously provided before and after the movement of the user object 601A.

[0125] In addition, in the first and second embodiments described above, the chat processing unit 35 can also provide a voice chat function. Furthermore, for example, in the second embodiment, if the user object 601 and the virtual camera move farther than a predetermined distance while the user object 601 is warping, the voice chat function may be stopped.

[0126] [Operation Flow] Figure 15 is a flowchart illustrating an example of the program's operation flow according to the third embodiment of this disclosure. The explanation will not be repeated regarding aspects common to the first or second embodiment, namely the movement of the user object 601, the change in the orientation of the virtual camera, and the warp movement of the user object 601. Furthermore, it is assumed that, similar to the flowchart shown in Figure 11, the acquisition of operation information from the operation detection unit 26 and the control of the hand object 605 are performed continuously.

[0127] Referring to Figures 2 and 15, first, the object control unit 33 determines whether or not a switching operation has been performed by the user based on the operation information from the operation detection unit 26 (step S41). If the object control unit 33 determines that no switching operation has been performed by the user (NO in step S41), it repeats the operation in step S41 each time it acquires operation information.

[0128] On the other hand, if the object control unit 33 determines that a switching operation has been performed by the user (YES in step S41), it switches from normal mode to camera position control mode (step S42).

[0129] Next, the object control unit 33 detects an operation on the operation object 604 based on the operation information from the operation detection unit 26. Then, the object control unit 33 moves the user object 601 in response to the operation on the operation object 604, for example, while the position and orientation of the virtual camera are fixed (step S43).

[0130] Furthermore, the chat processing unit 35, for example, disables the voice chat function and sends movement information to the server 20 indicating that the user object 601 is being moved in camera position control mode (step S44).

[0131] Next, the object control unit 33 determines whether the user has performed a switching operation again based on the new operation information from the operation detection unit 26 (step S45). If the user has not performed a switching operation (NO in step S45), the object control unit 33 continues the movement of the user object in camera position control mode (step S43).

[0132] On the other hand, if the user performs a switching operation (YES in step S45), the object control unit 33 switches from camera position control mode to normal mode (step S46). The object control unit 33 then moves the virtual camera to the vicinity of the user object 601 after it has been moved in camera position control mode (step S47). The chat processing unit 35 also restarts the voice chat function. The object control unit 33 then repeats the operations from step S41 onward each time it acquires operation information.

[0133] As described above, the system can switch between normal mode and camera position control mode depending on user input. When operating in camera position control mode, the system can reduce user motion sickness caused by the virtual camera following the user object, thereby improving user experience.

[0134] [Examples of virtual camera movement in camera position control mode] The object control unit 33 is not limited to a configuration that fixes the position and orientation of the virtual camera in camera position control mode. For example, the object control unit 33 may control the virtual camera to move in conjunction with the movement of the user object 601 in camera position control mode.

[0135] At this time, the object control unit 33 slows the movement speed of the virtual camera to be slower than the movement speed of the user object 601. This allows the user to see the movement path of the user object 601 while reducing changes in the virtual space image caused by the movement of the virtual camera, thereby reducing motion sickness for the user.

[0136] The elements included in each of the first to third embodiments of this disclosure can be combined with each other, and the resulting combinations also constitute a part of the embodiments shown in this disclosure. Furthermore, within the scope of the invention, the present invention allows for free combination of each component, modification of any component, substitution of any component, omission of any component, or addition of other components.

[0137] Furthermore, the processing flow described herein is merely an example, and the order and structure of each process may differ. Also, some processes described herein may not exist. In other words, the processing flow and specific decision processes may differ from those exemplified herein.

[0138] [Additional Notes] The contents of the embodiments of this disclosure are listed below. [assignment] The aim is to improve interest and appeal.

[0139] [Solution] (Item 1-1) Computers, A control means for operating operation objects and user objects placed in a virtual space based on user operations, It functions as an image processing means for displaying a virtual space image, which is an image of the virtual space based on a virtual camera placed in the virtual space. The control means is When an operation is performed on the operation object in a first direction, or when an operation is performed in a second direction after an operation in the first direction, the user object is moved according to the movement of the operation object. A program that, if the operation object is not performed in the first direction but is performed in the second direction, does not move the user object and changes the orientation of the virtual camera according to the operation of the operation object.

[0140] This configuration improves user experience because the target for moving user objects and the target for changing the orientation of the virtual camera are the same. Furthermore, since the target of manipulation is an object placed in the virtual space, there is no need for hardware such as controllers. Therefore, even inexperienced users can play easily and casually. This is particularly effective for virtual space images provided without the need for dedicated hardware.

[0141] (Item 1-2) The control means, when it detects the first action of the user, enables the operation on the operation object, as described in item 1-1. This configuration prevents unintentional manipulation of the manipulated object.

[0142] (Item 1-3) The program described in item 1-2, wherein the first action is a grasping or pressing action. This configuration allows for easy, one-handed operation on the target object.

[0143] (Items 1-4) The aforementioned operating object is stick-shaped, The control means is a program according to any one of items 1-1 to 1-3, which detects the tilting of the operating object in the first or second direction as an action of the operating object in the first or second direction, respectively. This configuration allows, for example, the user object's movement speed or the amount of change in the virtual camera's orientation to be altered according to the tilt of the manipulated object, thereby further improving user operability.

[0144] (Item 2-1) Computers, A control means for operating operation objects and user objects placed in a virtual space based on the user's operations in a predetermined area, It functions as an image processing means for displaying a virtual space image, which is an image of the virtual space based on a virtual camera placed in the virtual space. The control means, upon detecting a first operation by the user outside the predetermined area, places a location-specifying object in the virtual space that specifies the destination of the user object, and moves the user object to the destination in a specific manner that is faster than normal movement.

[0145] This configuration allows user objects to be moved to their destination in a short amount of time, thereby improving user operability. Furthermore, users can differentiate between normal movement and specific types of movement by changing the area of ​​movement. Therefore, there is no need for hardware such as controllers, making it easy and convenient for even inexperienced users to play. Additionally, the reduced movement time of user objects can lessen motion sickness caused by the virtual camera following the user object.

[0146] (Item 2-2) The control means is the program described in item 2-1, which instantaneously moves the user object to the destination as the movement in the specific mode. This configuration makes it possible to more effectively reduce user motion sickness caused by changes in the virtual space image when user objects are moved.

[0147] (Item 2-3) The program according to item 2-1 or item 2-2, wherein the control means can change the destination when it detects that the user has performed a second operation while continuing the first operation. This configuration allows the user to change the destination to any location they choose.

[0148] (Item 2-4) The first operation described above is an operation based on the first action of the user, The program described in item 2-3, wherein the second operation is an operation by the user based on a predetermined operation different from the first operation. In this way, it is possible to distinguish between the placement of a specified object and the change in the position of that specified object in response to changes in user behavior, thereby improving user engagement.

[0149] (Item 2-5) The first operation described above is the operation of the first part of the user, The program described in item 2-4, wherein the predetermined operation is an operation of a second body part of the user that is different from the first body part. With this configuration, for example, if the first part is the fingers of the hand and the second part is the elbow or wrist, the user can both place the position-specified object and change the position of that object with the movement of one hand.

[0150] (Item 3-1) Computers, A control means for operating operation objects, user objects, and virtual cameras placed in a virtual space based on user operations, It functions as an image processing means for displaying a virtual space image, which is an image of the virtual space based on the virtual camera, The control means is A first mode controls the movement of the user object and the virtual camera so that the virtual camera follows the movement of the user object, A program that can switch between a second mode, which controls the movement of the user object and the virtual camera such that the virtual space image displayed by the image processing means when the user object is moved is different from the virtual space image displayed when the virtual camera is controlled in the first mode.

[0151] This configuration, combined with the control of the second mode, reduces user motion sickness caused by the virtual camera following user objects, thereby improving user operability.

[0152] (Item 3-2) The program described in item 3-1, wherein the control means, after controlling the movement of the user object and the virtual camera in the second mode, terminates the second mode, and then moves the virtual camera to the vicinity of the user object after it has been moved. This configuration allows users to check the user object and its surroundings after movement using virtual space images.

[0153] (Item 3-3) The control means, in the second mode, sets the movement speed of the virtual camera slower than the movement speed of the user object, according to the program described in item 3-1 or item 3-2. This configuration allows users to see the movement path of user objects while minimizing changes in the virtual space image caused by the movement of the virtual camera, thereby reducing motion sickness in the user.

[0154] (Item 3-4) The control means, in the second mode, does not change the position and orientation of the virtual camera, as described in item 3-1 or item 3-2. This configuration further reduces changes in the virtual space image when user objects move, thereby more effectively reducing user motion sickness.

[0155] (Item 3-5) The aforementioned program, the computer, This chat processing means provides voice chat functionality to each user of a plurality of user objects located in the aforementioned virtual space. The chat processing means is a program according to any one of items 3-1 to 3-4, which stops the voice chat function when the movement of the user object and the virtual camera is controlled in the second mode.

[0156] Here, let's assume that the first user object and the second user object are located in the same virtual space. When the first user object moves in the second mode, the first user object and the virtual camera may move far apart, and the range visible from the position of the first user object and the range captured by the virtual camera, that is, the range visible to the first user operating the first user object, may differ significantly.

[0157] In this case, for example, even if the virtual space image displayed on the terminal device of the second user manipulating the second user object includes both the first and second user objects, the virtual space image displayed on the first user's terminal device will not include either the first or second user objects. In such a situation, if the first and second users engage in voice chat, the visible range of the first user as assumed by the second user may differ from the actual visible range of the first user, potentially causing a sense of incongruity.

[0158] In contrast, as described above, the configuration that disables the voice chat function when user objects are being moved in the second mode prevents the aforementioned sense of incongruity.

[0159] (Item 3-6) The chat processing means, when it stops providing the voice chat function, transmits to another device information indicating that the movement of the user object and the virtual camera is being controlled in the second mode, according to the program described in item 3-5. With this configuration, for example, based on the information sent from the computer of the first user (the user moving the user object in second mode), it is possible to display on the computer of the second user that the user object is moving in second mode. This allows the second user to understand why the voice chat function between them and the first user has been disabled.

[0160] (Item 3-7) The control means is a program according to any one of items 3-1 to 3-6, which transmits information indicating the position of the user object before it was moved to another device while controlling the movement of the user object and the virtual camera in the second mode.

[0161] With this configuration, for example, based on the information transmitted from the computer of the first user (the user moving the user object in second mode), the computer of the second user can display the user object being moved in second mode at its original position.

[0162] The position before this movement is, for example, around the first user's virtual camera, i.e., around the first user's viewpoint. Therefore, the second user can easily estimate the range visible from the first user by checking the position of the first user's user object displayed on the computer.

[0163] (Item 3-8) The aforementioned program, the computer, This chat processing means provides voice chat functionality to each user of a plurality of user objects located in the aforementioned virtual space. The chat processing means continues to provide the voice chat function before and after the control of the movement of the user object and the virtual camera in the second mode, as described in item 3-7.

[0164] This configuration allows voice chat between the first and second users to continue even if the first user's user object starts moving in second mode.

[0165] Furthermore, the solutions constructed in the above program may be adapted to the fields of devices, systems, methods, and media as appropriate.

[0166] Furthermore, the operations performed by the control unit 22 (see Figure 2) in the server 20 of each of the above embodiments may be performed by one processor or by multiple processors. For example, some or all of the operations performed by the control unit 22 may be performed by the terminal device 10 shown in Figure 1.

[0167] Furthermore, the above embodiments are merely illustrative examples to facilitate understanding of the present invention and are not intended to limit its interpretation. The present invention may be modified, improved, or have parts of its embodiments deleted without departing from its spirit, and it goes without saying that the present invention includes equivalents thereof. [Explanation of Symbols]

[0168] 1: Information processing system, 10, 10A, 10B: Terminal device, 13: Network, 14: Wireless router, 20: Server, 21: Communication unit, 22: Control unit, 23: Storage unit, 24: Memory, 25: Display unit, 26: Operation detection unit, 31: Space generation unit, 32: Arrangement unit, 33: Object control unit, 34: Image processing unit, 35: Chat processing unit, 41: Program, 42: User information, 601, 601A, 601B: User object, 602: Operation panel, 604: Operation object, 603: Menu button, 605, 605A, 605B: Hand object, 606: Switch button, 701: Position specification object, 702: Parabola

Claims

1. Computers, A control means for operating operation objects and user objects placed in a virtual space based on user operations, It functions as an image processing means for displaying a virtual space image, which is an image of the virtual space based on a virtual camera placed in the virtual space. The control means is When an operation is performed on the operation object in a first direction, or when an operation is performed in a second direction after an operation in the first direction, the user object is moved according to the movement of the operation object. A program that, if the operation object is not performed in the first direction but is performed in the second direction, does not move the user object and changes the orientation of the virtual camera according to the operation of the operation object.

2. The program according to claim 1, wherein the control means enables an operation on the operation object when it detects the first action of the user.

3. The program according to claim 2, wherein the first operation is a grasping operation or a pressing operation.

4. The aforementioned operating object is stick-shaped, The program according to claim 1 or 2, wherein the control means detects the tilting of the operating object in the first direction or the second direction as an action of the operating object in the first direction or the second direction, respectively.

5. An information processing system comprising one or more information processing devices, A control means for operating operation objects and user objects placed in a virtual space based on user operations, The system includes an image processing means for displaying a virtual space image, which is an image of the virtual space based on a virtual camera placed in the virtual space. The control means is When an operation is performed on the operation object in a first direction, or when an operation is performed in a second direction after an operation in the first direction, the user object is moved according to the movement of the operation object. An information processing system that, if the operation object is not performed in the first direction but is performed in the second direction, does not operate the user object and changes the orientation of the virtual camera according to the operation of the operation object.

Images