Information processing device, information processing method, and program

Abstract

[Problem] To enable a prediction result to be displayed in a manner that facilitates intuitive understanding. [Solution] An information processing device comprising a control unit that executes: processing for predicting the orientation of an object after a predetermined time has elapsed, said prediction being based on game information including at least a history of the movement of the object as operated by a player; and processing for displaying the predicted orientation of the object in a display format different from that of the current image of the object.

Description

Information processing device, information processing method, and program

[0001] The present disclosure relates to an information processing device, an information processing method, and a program.

[0002] In recent years, technologies have become known that provide virtual spaces in which various objects, such as 3D models or 2D images, are placed. Users can experience the virtual spaces using various types of information processing terminals, such as smartphones, head-mounted displays (HMDs), or PCs. Avatars (e.g., 3DCG presented based on 3D models) that are associated with and can be controlled by the users can be placed in the virtual spaces.

[0003] Furthermore, a large number of information processing terminals are connected to a server that controls such a virtual space. Each information processing terminal displays an image of the virtual space from the user's viewpoint based on information received from the server. Here, it is desirable that the movements of each avatar in the virtual space be synchronized between the information processing terminals. Patent Document 1 listed below discloses a technology that makes it easier to synchronize the movements of avatars between information processing terminals.

[0004] Japanese Patent Application Laid-Open No. 2018-196577

[0005] The above-mentioned Patent Document 1 mainly discloses prediction of hand positions in punching motions, but does not mention displaying prediction results in a way that makes them easy to understand intuitively.

[0006] Therefore, the present disclosure proposes an information processing device, an information processing method, and a program that are capable of displaying prediction results in a way that makes them easy to intuitively understand.

[0007] According to the present disclosure, there is provided an information processing device that includes a control unit that performs a process of predicting the posture of an object operated by a player after a predetermined time based on game information that includes at least a history of the movement of the object, and a process of displaying the predicted posture of the object in a display manner that differs from the current image of the object.

[0008] Furthermore, according to the present disclosure, there is provided an information processing method including: a processor predicting an attitude of an object controlled by a player after a predetermined time based on game information including at least a history of the movement of the object; and displaying the predicted attitude of the object in a display manner different from a current image of the object.

[0009] Furthermore, according to the present disclosure, a program is provided that causes a computer to function as a control unit that controls the display of an object's posture after a predetermined time, which posture is predicted based on game information including at least a history of the movement of the object operated by a player, in a display manner different from the current image of the object.

[0010] FIG. 11 is a diagram illustrating an overall configuration of an information processing system 1 according to an embodiment of the present disclosure. FIG. 12 is a block diagram illustrating an example of the configuration of a server 20 according to this embodiment. FIG. 13 is a block diagram illustrating an example of the configuration of an observer terminal 30 according to this embodiment. FIG. 14 is a diagram illustrating a first display example according to this embodiment. FIG. 15 is a diagram illustrating a second display example according to this embodiment. FIG. 16 is a diagram illustrating a third display example according to this embodiment. FIG. 17 is a diagram illustrating a fourth display example according to this embodiment. FIG. 18 is a diagram illustrating a fifth display example according to this embodiment. FIG. 19 is a flowchart illustrating a flow of overall operation processing of the information processing system 1 according to this embodiment. FIG. 19 is a flowchart illustrating operation processing of the observer terminal 30 according to this embodiment. FIG. 19 is a flowchart illustrating operation processing of displaying an operation prediction result in step S209 shown in FIG. 10. FIG. 20 is a diagram illustrating prediction timing according to a first modified example of this embodiment.

[0011] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0012] The explanation will be given in the following order: 1. Configuration 1-1. System configuration 1-2. Configuration of server 20 1-3. Configuration of observer terminal 30 2. Display examples 2-1. Basic display example of prediction results 2-2. Display example of operation prediction according to the number of avatars within the angle of view 2-3. Display example of operation prediction according to approach to a target avatar 2-4. Optional display control of operation prediction 3. Operation processing 3-1. Overall flow 3-2. Operation processing of observer terminal 30 4. Modifications 4-1. First modification 4-2. Second modification 4-3. Third modification 4-4. Fourth modification 5. Supplementary notes

[0013] 1. Configuration An information processing system according to an embodiment of the present disclosure will be specifically described with reference to the drawings.

[0014] <<1-1. System Configuration>> Figure 1 is a diagram showing the overall configuration of an information processing system 1 according to an embodiment of the present disclosure. The information processing system 1 according to this embodiment includes one or more player terminals 10 (10a to 10n), a server 20 (an example of an information processing device), and observer terminals 30 (30a to 30n) (an example of a user terminal).

[0015] In this embodiment, it is assumed that a sports game such as soccer is played in a metaverse (an example of a virtual space) controlled by a server 20. The server 20 reflects operation information transmitted from each player terminal 10 playing the game in each avatar (an example of an object) associated with each player in the metaverse. The avatar is, for example, 3DCG displayed based on a 3D model. The server 20 also controls the transmission of information for displaying game images to each observer terminal 30.

[0016] The observer terminal 30 is an information processing terminal (an example of a user terminal) used by an observer when watching (observing) a game. The observer terminal 30 is realized by, for example, a smartphone, a tablet terminal, a PC (personal computer), an HMD (head mounted display) that covers the entire field of view, a glasses-type display device, a projector, or the like. In this specification, the observer includes not only general viewers who watch the game, but also managers of game distribution and workers who decide the camera work for the distributed video.

[0017] Here, when observers are watching a sports game being played in the Metaverse, the server 20 according to this embodiment predicts the posture of each avatar participating in the game in real time and presents the predicted results on the video of the sports game being watched by the observers in an intuitive and easy-to-understand manner. This allows, for example, an operator who determines the camerawork for the distributed video to prepare for the next camerawork operation. Furthermore, general viewers can enjoy watching the game more by knowing the predicted next development.

[0018] The overview of the information processing system 1 according to an embodiment of the present disclosure has been described above. Next, the specific configuration of each device included in the information processing system 1 according to this embodiment will be described with reference to the drawings.

[0019] 2 is a block diagram showing an example of the configuration of the server 20 according to this embodiment. As shown in FIG. 2, the server 20 includes a communication unit 210, a control unit 220, and a storage unit 230.

[0020] (Communication Unit 210) The communication unit 210 has a transmission unit that transmits data to an external device and a reception unit that receives data from an external device. The communication unit 210 according to this embodiment may be communicatively connected to an external device or the Internet using, for example, a wired or wireless LAN (Local Area Network), Wi-Fi (registered trademark), Bluetooth (registered trademark), a mobile communication network (LTE (Long Term Evolution), 4G (fourth generation mobile communication system), 5G (fifth generation mobile communication system)), or the like.

[0021] (Control Unit 220) The control unit 220 functions as an arithmetic processing unit and a control device, and controls the overall operation of the server 20 in accordance with various programs. The control unit 220 is realized by an electronic circuit such as a CPU (Central Processing Unit) or a microprocessor. The control unit 220 may also include a ROM (Read Only Memory) that stores the programs to be used, arithmetic parameters, etc., and a RAM (Random Access Memory) that temporarily stores parameters that change as appropriate.

[0022] The control unit 220 according to this embodiment can also function as a game information encoding unit 221, a player information encoding unit 222, an encoded information combining unit 223, a posture prediction unit 224, a presentation control unit 225, and an encoded information transmission control unit 226.

[0023] The game information encoding unit 221 encodes game information (an example of feature extraction) that is progressing in real time. The game information includes at least the movement history of each avatar (object) controlled by the player. The movement history of each avatar is at least one of the following: the posture history of each avatar (specifically, the history of coordinate information of joint positions for each frame) and the operation history of the player with respect to the avatar (specifically, the history of operation items such as running, passing, shooting, intercepting a pass, and stopping for each frame). The game information encoding unit 221 encodes, for example, the operation history of all players and the posture history of all avatars from the start of the game. The game information may further include information such as the ball position history, score, and game time (the elapsed time since the start of the game). The game information encoding unit 221 may encode game information that also includes this information. The encoding method is not particularly limited. For example, the game information encoding unit 221 may convert the above-mentioned game information into vectors using a trained model obtained by machine learning such as a neural network.

[0024] The player information encoding unit 222 encodes player information (an example of feature extraction) related to the players controlling each avatar participating in the game. The player information includes at least one of a player's field of view video history and a sound history in the game. The player's field of view video history provides information on the movements of other avatars around the avatar controlled by the player. The sound history is a history of the audio heard by the player. For example, the sound history provides information on the speech of other players and the cheers of the audience. The player information may further include the position (role) in the game of the avatar controlled by the player and ability scores. The player information encoding unit 222 may encode player information including these pieces of information. The encoding method is not particularly limited. For example, the player information encoding unit 222 may convert the above-described player information into a vector using a trained model obtained by machine learning, such as a neural network.

[0025] The coded information combining unit 223 performs processing to combine the information coded by the game information coding unit 221 and the information coded by the player information coding unit 222 .

[0026] The posture prediction unit 224 performs processing to predict the posture of the avatar after a predetermined time. The predetermined time may be, for example, several seconds, such as 2 to 5 seconds. Examples of postures include a running posture (action), a passing posture (action), a shooting posture (action), a pass-cutting posture (action), and a stopping posture (action). The predicted posture may also include a position. That is, the posture prediction unit 224 may predict the position and posture of the avatar after a predetermined time. The posture prediction unit 224 may perform processing to predict the posture of the avatar after a predetermined time based on game information including at least the movement history of each avatar controlled by each player. Furthermore, the posture prediction unit 224 may perform processing to predict the posture of the avatar after a predetermined time based on player information. In this embodiment, by using player information including peripheral information that may affect the player's operation decisions, such as the movements of other avatars in the vicinity and the voices of other players, irregular movements of the avatar based on the surrounding circumstances can also be predicted. The posture prediction unit 224 may output the posture with the highest prediction probability as the prediction result (also referred to as the posture prediction result).

[0027] In this embodiment, the posture prediction unit 224 predicts the posture of each avatar after a predetermined time based on the coded information obtained by combining the coded information of the game information and the coded information of the player information.

[0028] The posture prediction unit 224 may predict the posture of each avatar after a predetermined time using a trained prediction model obtained by machine learning such as a neural network. The posture prediction unit 224 may calculate the joint positions (coordinate information) of the entire body of the avatar after the predetermined time as a prediction of the posture of the avatar after the predetermined time.

[0029] More specifically, the posture prediction unit 224 predicts coded information after a predetermined time using a trained prediction model based on coded information obtained by combining coded information of game information and coded information of player information, and converts the predicted coded information into joint positions (coordinate information) of the entire body for each avatar using the trained model. Examples of joint positions include the coordinate positions of the head, neck, shoulders, elbows, wrists, waist, knees, ankles, and toes.

[0030] In this manner, in this embodiment, the postures of all avatars after a predetermined time can be predicted based on the game information and the player information. Note that the posture prediction unit 224 may also predict the postures of all avatars after a predetermined time based only on the game information, without using the player information. Whether or not to use the player information for the prediction may be set in advance by the administrator.

[0031] The above-described method of posture prediction is an example, and the present disclosure is not limited to this.

[0032] The presentation control unit 225 has a function of controlling the presentation of the prediction results (posture information (coordinate information) of each avatar after a predetermined time) to the observer. Specifically, the presentation control unit 225 controls the transmission of the prediction results from the communication unit 210 to the observer terminal 30 via the network 40.

[0033] Here, the predicted avatar postures may be presented to the observer in a display format different from that of the current avatar image. Display examples will be described later with reference to FIGS. 4 to 8. The presentation control unit 225 controls the transmission of information for displaying the predicted avatar postures in a display format different from that of the current avatar to the observer terminal 30 (user terminal).

[0034] The coded information transmission control unit 226 controls the transmission of the combined coded information to the observer terminal 30. The coded information is used for operation prediction performed in the observer terminal 30.

[0035] (Storage Unit 230) The storage unit 230 is realized by a ROM that stores programs and calculation parameters used in the processing of the control unit 220, and a RAM that temporarily stores parameters that change as needed.

[0036] Although the configuration of the server 20 has been specifically described above, the configuration of the server 20 according to the present disclosure is not limited to the example shown in Fig. 2. For example, the server 20 does not necessarily have to have all of the components shown in Fig. 2. Furthermore, the server 20 may be realized by multiple devices.

[0037] <<1-3. Configuration of the observer terminal 30>> Fig. 3 is a block diagram showing an example of the configuration of the observer terminal 30 according to this embodiment. As shown in Fig. 3, the observer terminal 30 includes a communication unit 310, a control unit 320, an operation input unit 330, a display unit 340, an audio output unit 350, and a storage unit 360.

[0038] (Communication Unit 310) The communication unit 310 has a transmission unit that transmits data to an external device and a reception unit that receives data from an external device. The communication unit 310 according to this embodiment may be communicatively connected to an external device or the Internet using, for example, a wired or wireless LAN (Local Area Network), Wi-Fi (registered trademark), Bluetooth (registered trademark), a mobile communication network (LTE (Long Term Evolution), 4G (fourth generation mobile communication system), 5G (fifth generation mobile communication system)), or the like.

[0039] (Control Unit 320) The control unit 320 functions as an arithmetic processing unit and a control device, and controls the overall operation of the observer terminal 30 in accordance with various programs. The control unit 320 is realized by an electronic circuit such as a CPU (Central Processing Unit) or a microprocessor. The control unit 320 may also include a ROM (Read Only Memory) that stores the programs to be used, arithmetic parameters, etc., and a RAM (Random Access Memory) that temporarily stores parameters that change as appropriate.

[0040] The control unit 320 according to this embodiment can also function as an operation prediction unit 321 and a display control unit 322 .

[0041] The operation prediction unit 321 performs processing to predict operations that a player operating an avatar will perform after a predetermined time. The operation prediction unit 321 may perform processing to predict operations that a player will perform on an avatar after a predetermined time, based on game information including at least a history of movements of each avatar operated by each player. Furthermore, the operation prediction unit 321 may perform processing to predict operations that a player will perform on an avatar after a predetermined time, based further on player information.

[0042] In this embodiment, the operation prediction unit 321 predicts the operation that each player will perform on their avatar after a predetermined time based on the coded information that combines the coded information of the game information and the coded information of the player information sent from the server 20.

[0043] The operation prediction unit 321 may predict an operation that each player will perform on an avatar after a predetermined time using a trained prediction model obtained by machine learning such as a neural network. The operation prediction unit 321 outputs operation items such as run, pass, and stop as predictions of operations that each player will perform on an avatar after a predetermined time. The output operation items are accompanied by predicted probabilities.

[0044] More specifically, the operation prediction unit 321 predicts the coded information after a predetermined time using a trained prediction model based on the coded information that combines the coded information of the game information and the coded information of the player information, and converts the predicted coded information into an operation item (with a predicted probability) using the trained model.

[0045] In this way, in this embodiment, operations of all players after a predetermined time can be predicted based on the game information and player information. Note that the operation prediction unit 321 may also predict operations of all players after a predetermined time based only on the game information, without using the player information. Whether or not to use player information for prediction may be set in advance by the observer (user).

[0046] Furthermore, whether or not the operation prediction unit 321 will perform operation prediction may be set in advance by the observer (user).

[0047] The above-described method for operation prediction is an example, and the present disclosure is not limited to this.

[0048] The display control unit 322 controls the display of the current avatars and also the display of the prediction results on the display unit 340. The prediction results include posture information (coordinate information) of each avatar after a predetermined time, which is transmitted from the server 20, and operation information (including prediction probability) of each player after a predetermined time, which is predicted by the operation prediction unit 321.

[0049] The display control unit 322 controls the display of each predicted avatar posture in a display mode different from that of the current avatar, based on information transmitted from the server 20 for displaying each predicted avatar posture in a display mode different from that of the current avatar (for example, predicted avatar image R1 shown in FIG. 4 ). The display control unit 322 may display the posture with the highest prediction probability among one or more postures predicted by the server 20 in a display mode different from that of the current avatar. Furthermore, the display control unit 322 may display the predicted avatar posture so that it at least partially overlaps with the current avatar, may display it near the current avatar (at a certain distance from the current avatar), or may display it at a position included in the predicted posture.

[0050] Furthermore, the display control unit 322 controls the display of operation items predicted by the operation prediction unit 321 for each player after a predetermined time in association with each avatar (see FIGS. 4 and 5). The display control unit 322 may also display the predicted operation items together with their predicted probabilities. The display control unit 322 may also display the predicted operation items in a display mode according to the predicted probabilities. Details of display examples will be described later with reference to FIGS. 4 to 8.

[0051] (Operation Input Unit 330 and Display Unit 340) The operation input unit 330 accepts operation input from a user (observer) and outputs the input information to the control unit 320. The operation input unit 330 may be realized by a mouse, a keyboard, a switch, a button, or the like. The display unit 340 displays an image of a game being played in the Metaverse. The display unit 340 may be a display panel such as a liquid crystal display (LCD) or an organic electroluminescence (EL) display. The display unit 340 may be formed to cover the user's field of vision, for example.

[0052] The operation input unit 330 and the display unit 340 may be provided separately or integrally. The operation input unit 330 may be a touch sensor stacked on the display unit 340 (for example, a panel display).

[0053] (Audio Output Unit 350) For example, when the observer terminal 30 is realized by an HMD, the audio output unit 350 is configured as headphones to be worn on the user's head and plays back audio signals. The audio output unit 350 is not limited to a headphone type, and may be configured as earphones or a bone conduction speaker.

[0054] (Storage Unit 360) The storage unit 360 is realized by a ROM that stores programs and calculation parameters used in the processing of the control unit 320, and a RAM that temporarily stores parameters that change as needed.

[0055] The configuration of the observer terminal 30 has been specifically described above, but the configuration of the observer terminal 30 according to the present disclosure is not limited to the example shown in Fig. 3. For example, the observer terminal 30 does not necessarily have to have all of the components shown in Fig. 3. Furthermore, the observer terminal 30 may be realized by a plurality of devices.

[0056] <<2. Display Examples>> Next, display examples of prediction results on the observer terminal 30 according to this embodiment will be described with reference to FIGS.

[0057] (2-1. Basic Display Example of Prediction Results) FIG. 4 illustrates a first display example according to this embodiment. Display example 400 shown in FIG. 4 displays avatar P1 and predicted avatar R1, which corresponds to the predicted avatar posture (prediction result) after a predetermined time. Predicted avatar R1 may be displayed in a different display mode from avatar P1. Furthermore, predicted avatar R1 may be displayed at a position included in the predicted posture. For example, predicted avatar R1 may be displayed as a transparent image of the avatar, a silhouette image of the avatar, an outline image of the avatar, or a skeleton image formed by dots indicating joint positions and lines connecting the dots. Furthermore, predicted avatar R1 may be a still image or a moving image (animation). Furthermore, the display duration of predicted avatar R1 is not particularly limited. For example, predicted avatar R1 may be displayed until the next posture prediction is performed, until the predicted time (a predetermined time from the present) is reached, or until a specified time, such as two or three seconds, has elapsed. The predicted avatar image R1 can be displayed by fitting an avatar (3D model) to the posture of the avatar after a predetermined time indicated by the prediction result, i.e., the positions of each joint (coordinate information).

[0058] In addition, the observer terminal 30 may display predicted avatar images of all avatars displayed within the field of view (within the observer's field of view) of the game video displayed on the display unit 340, or may display predicted avatar images of at least the avatar holding the ball and the avatars around that avatar.

[0059] 4, an image 401 showing the predicted operation item (prediction result) of the player after a predetermined time is displayed in association with the predicted avatar image R1 of the avatar operated by the player. The image 401 also displays the predicted probability of the operation item.

[0060] (2-2. Example of Displaying Operation Predictions According to the Number of Avatars Within the Field of View) The observer terminal 30 may display operation predictions (predictions of operation items that the player will perform after a predetermined time) when the number of avatars within the field of view (within the observer's field of view) of the game video displayed on the display unit 340 is equal to or less than a set number, and may not display operation predictions when the number exceeds the set number. This makes it possible to avoid the risk of reduced visibility of the display screen due to an increase in the number of displayed images.

[0061] Display control of operation predictions according to the number of avatars when the set number is "2," for example, will be described with reference to FIGS. 5 and 6 . FIG. 5 illustrates a second display example according to this embodiment. In display example 410 illustrated in FIG. 5 , the number of avatars (avatar images P1 and P2) included within the field of view is equal to or less than the set number "2." Therefore, operation prediction images 412a and 412b are displayed in association with the predicted avatar images R1 and R2, respectively. The method of displaying operation predictions is not particularly limited. For example, the predicted probability may be displayed numerically as illustrated in FIG. 4 or graphically as illustrated in FIG. 5 . Furthermore, observer terminal 30 may display operation items with larger characters or in a more conspicuous color, depending on the operation item's predicted probability. Furthermore, operation items are not limited to being displayed as text. Observer terminal 30 may, for example, display icon images representing the operation items.

[0062] 6 is a diagram showing a third display example according to this embodiment. In display example 420 shown in FIG. 6, the number of avatars (avatar images P1, P2, and P3) included within the field of view exceeds the set number "2," so no operation predictions are displayed. In the example shown in FIG. 6, only predicted avatar images R1, R2, and R3 are displayed as the prediction results.

[0063] (2-3. Example of display of operation predictions according to approach to the focused avatar) When another avatar is approaching the focused avatar that is included within the field of view of the game video displayed on the display unit 340 (within the field of view of the observer), the observer terminal 30 may perform control so as to increase the number of presented operation prediction candidates corresponding to the other avatar, and when the other avatar is farther away, to decrease the number of presented operation prediction candidates displayed in association with the other avatar.

[0064] An example of a featured avatar is an avatar holding a ball.

[0065] As a display example when another avatar is approaching the avatar of interest, the above-mentioned Fig. 5 is used. As shown in Fig. 5, in the predicted operation image 412b corresponding to the avatar P2 (predicted avatar image R2) approaching the avatar P1 holding the ball, the number of operation items presented is increased to the same number as in the predicted operation image 412a corresponding to the avatar P1 (predicted avatar image R1) holding the ball. On the other hand, a display example when the avatar is far from the avatar of interest will be described with reference to Fig. 7.

[0066] Fig. 7 is a diagram showing a fourth display example according to this embodiment. In display example 430 shown in Fig. 7, the number of presented operation item candidates is reduced in predicted operation images 430b and 430c corresponding to avatars P2 and P3 (predicted avatars R2 and R3) that are at least a certain distance away from avatar P1 holding the ball, compared to the case shown in Fig. 5. Because avatars that are far away from the avatar of interest attract less attention, reducing the number of presented operation item candidates below a certain number can both improve the presentation of the prediction results and make them easier to view.

[0067] The threshold value used to determine whether the avatars are approaching or moving away may be set in advance.

[0068] (2-4. Optional Display Control of Operation Prediction) Whether or not to display the operation prediction can be arbitrarily controlled in accordance with an operation by an observer (user). The observer can collectively control whether or not to display the operation prediction for all avatars, or can individually control it.

[0069] Fig. 8 is a diagram showing a fifth display example according to this embodiment. Display example 440 shown in Fig. 8 is a display screen when an observer performs an operation on observer terminal 30 without displaying the operation prediction, as compared to display example 410 shown in Fig. 5. In the example shown in Fig. 8, only predicted avatar images R1 and R2 corresponding to avatar images P1 and P2 are displayed as the prediction results.

[0070] The observer terminal 30 may be automatically configured to present predicted operations for the players (avatars) of the supported team and the supported players (avatars) if the pre-set profile information of the observer includes information about the support team and the support player.

[0071] Furthermore, when the observer terminal 30 does not present an operation prediction, it can skip the operation prediction process by the operation prediction unit 321, thereby reducing the amount of calculation.

[0072] <<3. Operational Processing>> Next, the operational processing of this embodiment will be described.

[0073] <3-1. Overall Flow> FIG. 9 is a flowchart showing the overall flow of operation processing of the information processing system 1 according to this embodiment.

[0074] 9, the server 20 first acquires game information for a game being held in the Metaverse (step S103), and then performs an encoding process for the game information (step S106). The game information includes, for example, the posture history of each avatar and the operation history of the player transmitted from the player terminal 10.

[0075] Furthermore, when the server 20 acquires player information for a game being held in the Metaverse (step S109), it performs encoding processing of the player information (step S112). The player information includes, for example, the player's field of view video history and sound history.

[0076] Next, the server 20 combines the coded information of the game information with the coded information of the player information (step S115).

[0077] Next, the server 20 predicts the posture (information (coordinate information) of the joint positions of the entire body) of each avatar after a predetermined time based on the combined coded information (step S118).

[0078] Then, the server 20 controls the observer terminal 30 so that the predicted avatar posture is displayed (presented) in a display mode different from the current avatar image (step S121).

[0079] Furthermore, the server 20 transmits to the observer terminal 30 the coded information (combined coded information) used for operation prediction performed in the observer terminal 30 (step S124).

[0080] <3-2. Operational Processing of the Observer Terminal 30> FIG. 10 is a flowchart showing the operational processing of the observer terminal 30 according to this embodiment.

[0081] As shown in FIG. 10, first, the observer terminal 30 displays an image of a game being held in the metaverse on the display unit 340 based on the information transmitted from the server 20 (step S203).

[0082] Next, the observer terminal 30 displays the posture prediction result transmitted from the server 20 on the game video (step S206). That is, the observer terminal 30 displays the posture of the current avatar shown in the game video after a predetermined time in an image that is displayed in a different manner from the image of the current avatar.

[0083] The observer terminal 30 also predicts an operation based on the coded information of the game information and player information transmitted from the server 20, and displays the operation prediction result on the game image (step S206). The display process of the operation prediction result will be described later with reference to FIG. 11.

[0084] The observer terminal 30 then repeats steps S203 to S209 above until the display of the game video is completed (step S212).

[0085] FIG. 11 is a flowchart showing the operational process of displaying the operation prediction result in step S209 shown in FIG.

[0086] As shown in FIG. 11, first, the observer terminal 30 receives coded information of game information and player information from the server 20 (step S303).

[0087] Next, the observer terminal 30 starts the process of predicting an operation based on the coded information (step S306).

[0088] Next, the observer terminal 30 checks whether or not to display the operation prediction (step S309). The setting of whether or not to display the operation prediction can be set in advance by the observer.

[0089] Next, if the setting for whether or not to display the operation prediction is set to "none" (step S309 / No), the observer terminal 30 stops the operation prediction process (step S312). In this case, the operation prediction result is not displayed (step S315).

[0090] On the other hand, if the setting for displaying or not displaying operation predictions is set to "Yes" (step S309 / Yes), the observer terminal 30 checks whether any avatar is approaching the ball-carrying avatar (step S318). Note that the observer terminal 30 may determine whether any other avatar is approaching the ball-carrying avatar based on whether the Euclidean distance between the ball-carrying avatar and the other avatar, calculated based on the waist position (coordinate information) of each avatar, exceeds a predetermined threshold.

[0091] If there is an approaching avatar (step S318 / Yes), the observer terminal 30 increases the number of presentation candidates for the predicted operation results of the approaching avatar (step S321). In this embodiment, it is assumed that the number of presentation candidates for the predicted operation results of avatars other than the ball-holding avatar is set to a small number in advance.

[0092] Next, if there is no approaching avatar (step S318 / No), or after increasing the number of presented candidates for the operation prediction results of the approaching avatar (step S321), the observer terminal 30 checks whether the number of avatars within the observer's field of view (within the field of view of the game image displayed on the display unit 340) is less than the set number (step S324).

[0093] If the number is equal to or less than the set number (step S324 / Yes), the observer terminal 30 displays the operation prediction results of all avatars that are within the field of view of the game video displayed on the display unit 340 (step S327).

[0094] On the other hand, if the number is greater than the set number (step S324 / No), the observer terminal 30 hides the predicted operation results of all avatars that appear within the field of view of the game video displayed on the display unit 340 (step S315). Note that the observer terminal 30 may display the predicted operation results only for the avatar in possession of the ball, even if the number of avatars that appear within the field of view of the game video is greater than the set number. Furthermore, the observer terminal 30 may display the predicted operation results for the avatar in possession of the ball in situations where the game is moving, such as during a set play, a play in front of the goal, or when another avatar approaches the avatar in possession of the ball.

[0095] 4. Modifications Next, modifications of this embodiment will be described.

[0096] <<4-1. First Modification>> Fig. 12 is a diagram illustrating prediction timing according to a first modification of this embodiment. The upper part of Fig. 12 shows prediction timing in a normal scene in a game (a scene other than a scene where the match is in action). As shown in the upper part of Fig. 12, in a normal scene, the observer terminal 30 makes predictions at timings where the start of predictions does not overlap with each other.

[0097] The lower part of Fig. 12 shows prediction timings in scenes where the game is in action. Scenes where the game is in action include, for example, set plays, plays in front of the goal, and scenes where a play is taking place in which another avatar approaches an avatar in possession of the ball. In scenes where the game is in action, the observer terminal 30 may increase the frequency of prediction updates, that is, perform predictions at timings where the start of predictions overlap, as shown in the lower part of Fig. 12. Furthermore, in scenes where the game is in action, the observer terminal 30 may increase the number of presented candidates for predicted operations corresponding to the avatar in possession of the ball.

[0098] <<4-2. Second Modification>> In the above-described embodiment, it has been described that operation prediction is performed by the observer terminal 30, but the present disclosure is not limited to this, and operation prediction may be performed by the server 20. In this case, the server 20 may transmit the operation prediction result to the observer terminal 30.

[0099] <<4-3. Third Modification>> In the embodiment described above, it has been described that the posture prediction unit 224 predicts the posture of the avatar after a predetermined time, and that the operation prediction unit 321 predicts the operation of the player after a predetermined time, but this “predetermined time” can be dynamically changed.

[0100] For example, the posture prediction unit 224 or the operation prediction unit 321 may appropriately make a prediction after a predetermined time period set in advance, depending on the game situation. Examples of the game situation include the game time (the time elapsed since the start of the game, the time remaining until the end of the game), the phase of the game, etc. For example, the posture prediction unit 224 or the operation prediction unit 321 may set the "predetermined time period" in the prediction to be shorter than that in a normal phase in a game where the game is moving forward, such as a set play, a play in front of the goal, when another avatar approaches the ball-carrying avatar, or when an avatar on the same team as the ball-carrying avatar is waiting for a pass in front of the goal. On the other hand, when the game is not moving forward immediately, such as when there are no other avatars around the ball-carrying avatar, the "predetermined time period" in the prediction may be longer than that in a normal phase.

[0101] The posture prediction unit 224 or the operation prediction unit 321 may also perform predictions after a predetermined time period set in advance depending on the avatar or team. For example, the posture prediction unit 224 or the operation prediction unit 321 may shorten the "predetermined time period" in the prediction for a team or combination of teams whose game progresses quickly. The posture prediction unit 224 or the operation prediction unit 321 may also shorten the "predetermined time period" in the prediction for a specific avatar in a player or position who is leading the game progress compared to other avatars. Furthermore, the posture prediction unit 224 or the operation prediction unit 321 may shorten the "predetermined time period" in the posture prediction or operation prediction for the avatar when the specific avatar has possession of the ball or when the ball approaches within a certain distance from the specific avatar.

[0102] <<4-4. Fourth Modification>> In the above-described embodiment, the posture prediction unit 224 outputs the posture with the highest prediction probability from among one or more predicted postures as a prediction result, and the display control unit 322 displays the posture with the highest prediction probability in a display mode different from the current avatar. However, the present disclosure is not limited to this. For example, the posture prediction unit 224 may output a predetermined specific posture as a prediction result if the specific posture has a predetermined prediction probability or higher, and the display control unit 322 may display only the specific posture. An example of the specific posture is a shooting posture (action). Note that multiple specific postures may be set. Furthermore, a specific posture may be set for each observer terminal 30. In this case, the posture prediction unit 224 outputs the one or more predicted postures and their respective prediction probabilities as prediction results, and the prediction results are transmitted from the server 20 to the observer terminal 30. Then, the display control unit 322 displays only a predetermined specific posture from among the one or more predicted postures if the predetermined specific posture has a predetermined prediction probability or higher.

[0103] The posture prediction unit 224 may also change the "predetermined time" for prediction depending on the type of specific posture predetermined for the display target. The posture prediction unit 224 may also change the "predetermined time" for prediction depending on the type of specific posture and the game situation. For example, when the specific posture to be displayed is a shooting motion and there are no other avatars around the ball-holding avatar, the posture prediction unit 224 may lengthen the "predetermined time" and predict the shooting motion of the ball-holding avatar early on as the ball-holding avatar moves near the goal. In this case, the display control unit 322 may continuously display a predicted avatar image showing the predicted posture of the avatar near the current avatar. On the other hand, when there are many avatars from the opposing team around the ball-holding avatar, it is expected that the shooting motion will be predicted with a certain probability or higher once one of the avatars in possession of the ball approaches the goal, and therefore the posture prediction unit 224 may shorten the "predetermined time."

[0104] 5. Supplementary Information Although preferred embodiments of the present disclosure have been described above in detail with reference to the accompanying drawings, the present technology is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modified or altered examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0105] For example, one or more computer programs can be created for hardware such as a CPU, ROM, and RAM built into the above-described server 20 or observer terminal 30 to perform the functions of the server 20 or observer terminal 30. Also provided is a computer-readable storage medium storing the one or more computer programs.

[0106] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that will be apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.

[0107] The present technology may also be configured as follows. (1) An information processing device including a control unit that performs the following processes: predicting a posture of an object operated by a player after a predetermined time based on game information including at least a history of the movement of the object; and displaying the predicted posture of the object in a display mode different from a current image of the object. (2) The information processing device according to (1), wherein the history of the movement of the object includes at least one of a posture history of the object and an operation history of the player operating the object. (3) The information processing device according to (1) or (2), wherein the game information further includes information on the position of a ball used in the game. (4) The information processing device according to any one of (1) to (3), wherein the control unit predicts postures of all objects after a predetermined time based on game information including at least a history of the movement of all objects in the game. (5) The information processing device according to any one of (1) to (4), wherein the control unit extracts feature amounts of the movements of the objects in the game from the game information and predicts the postures of the objects after a predetermined time using a trained prediction model. (6) The information processing device according to (5), wherein the control unit calculates joint positions of the entire body of the object after a predetermined time as a prediction of the posture of the object after a predetermined time. (7) The information processing device according to any one of (1) to (6), wherein the control unit predicts the posture of the object after a predetermined time based on the game information and player information about the player. (8) The information processing device according to (7), wherein the player information includes at least one of a history of field of view images and a sound history of the player in the game. (9) The information processing device according to (8), wherein movements of other objects in the vicinity of the object are obtained from the history of field of view images of the player, and speech sounds of other players are obtained from the sound history.(10) The information processing device according to any one of (7) to (9), wherein the control unit predicts a posture of the object after a predetermined time using a trained prediction model based on feature amounts of movement of the object in the game extracted from the game information and feature amounts of information about the periphery of the object in the game extracted from the player information. (11) The information processing device according to any one of (1) to (10), wherein the control unit transmits to a user terminal information for displaying the predicted posture of the object in a display mode different from a current image of the object. (12) The information processing device according to (11), wherein the control unit transmits to the user terminal the game information used also for predicting an operation to be performed by the player operating the object after a predetermined time on the user terminal. (13) The information processing device according to any one of (1) to (11), wherein the control unit further predicts an operation to be performed by the player operating the object after a predetermined time on the basis of the game information, and displays an item of the predicted operation of the player in association with the object. (14) The information processing device according to (13), wherein the control unit displays one or more predicted operation items of the player together with predicted probabilities. (15) The information processing device according to (13) or (14), wherein the control unit displays one or more predicted operation items of the player in a display mode according to the predicted probabilities. (16) The information processing device according to any one of (13) to (15), wherein the control unit transmits information for displaying the predicted operation items of the player in association with the object to a user terminal. (17) The information processing device according to any one of (1) to (16), wherein the control unit dynamically changes the predetermined time. (18) An information processing method, comprising: a processor predicting a posture of the object after a predetermined time based on game information including at least a history of movements of the object controlled by a player; and displaying the predicted posture of the object in a display mode different from a current image of the object.(19) A program that causes a computer to function as a control unit that controls displaying, in a display mode different from a current image of the object, a posture of the object after a predetermined time, which is predicted based on game information including at least a history of the movement of the object operated by a player. (20) The program described in (19), wherein the control unit, when displaying, in association with the object, an operation item of the player operating the object after a predetermined time, which is predicted based on the game information, determines whether or not to display it based on a state of each object in a user's field of view image in a virtual space.

[0108] REFERENCE SIGNS LIST 1 Information processing system 10 Player terminal 20 Server 210 Communication unit 220 Control unit 221 Game information encoding unit 222 Player information encoding unit 223 Encoded information combining unit 224 Posture prediction unit 225 Presentation control unit 226 Encoded information transmission control unit 230 Storage unit 30 Observer terminal 310 Communication unit 320 Control unit 321 Operation prediction unit 322 Display control unit 330 Operation input unit 340 Display unit 350 Audio output unit 360 Storage unit 40 Network

Claims

1. An information processing device comprising a control unit that performs the following processes: predicting the posture of an object operated by a player after a predetermined time based on game information including at least a history of the object's movement; and displaying the predicted posture of the object in a display manner different from the current image of the object.

2. An information processing device according to claim 1, wherein the object's movement history includes at least one of an attitude history of the object and an operation history of the player operating the object.

3. The information processing device according to claim 1, wherein the game information further includes information on the position of a ball used in the game.

4. The information processing device according to claim 1, wherein the control unit predicts the postures of all objects after a predetermined time based on game information including at least a history of the movements of all objects in the game.

5. The information processing device according to claim 1, wherein the control unit extracts features of the object's movement in the game from the game information and predicts the object's posture after a predetermined time using a trained prediction model.

6. The information processing device according to claim 5, wherein the control unit calculates positions of all joints of the object after a predetermined time as a prediction of the posture of the object after the predetermined time.

7. The information processing device according to claim 1, wherein the control unit predicts the posture of the object after a predetermined time based on the game information and player information relating to the player.

8. An information processing device according to claim 7, wherein the player information includes at least one of a history of a visual image and a sound history of the player in a game.

9. An information processing device according to claim 8, wherein the movements of other objects in the vicinity of the object are obtained from the history of the player's field of view images, and the speech sounds of other players are obtained from the sound history.

10. The information processing device described in claim 7, wherein the control unit predicts the posture of the object after a predetermined time using a learned prediction model based on features of the object's movement in the game extracted from the game information and features of information about the object's surroundings in the game extracted from the player information.

11. The information processing device according to claim 1, wherein the control unit transmits to a user terminal information for displaying the predicted posture of the object in a display manner different from a current image of the object.

12. The information processing device according to claim 11, wherein the control unit transmits to the user terminal the game information that is also used to predict the operation of the player operating the object performed on the user terminal after a predetermined time.

13. The information processing device according to claim 1, wherein the control unit further predicts an operation of the player operating the object after a predetermined time based on the game information, and displays an item of the predicted operation of the player in association with the object.

14. The information processing device according to claim 13, wherein the control unit displays one or more predicted operation items of the player together with predicted probabilities.

15. The information processing device according to claim 13, wherein the control unit displays one or more predicted operation items of the player in a display mode according to the predicted probability.

16. The information processing device according to claim 13, wherein the control unit transmits to a user terminal information for displaying the predicted items of the player's operations in association with the object.

17. The information processing device according to claim 1, wherein the control unit dynamically changes the predetermined time period.

18. An information processing method comprising: a processor predicting a posture of an object controlled by a player after a predetermined time based on game information including at least a history of the object's movement; and displaying the predicted posture of the object in a display manner different from a current image of the object.

19. A program that causes a computer to function as a control unit that performs control to display the posture of an object controlled by a player at a predetermined time later, predicted based on game information including at least the movement history of the object, in a display mode different from the current image of the object.

20. The program described in claim 19, wherein when the control unit displays the operation items of the player operating the object after a predetermined time period, which are predicted based on the game information, in association with the object, the control unit determines whether or not to display them depending on the state of each object in the user's field of view image in the virtual space.

Images