Information processing device, server device, and video display method

Abstract

An information processing device 10 has the function of displaying video to which metadata has been set, information indicating an action that occurs in the video and the occurrence time of the action being associated in said metadata. A receiving unit 142 receives input of a character string indicating an action. An action identification unit 144 identifies a plurality of occurrence times of the action related to the received character string. A display processing unit 114 displays a highlight video obtained by linking together a plurality of partial videos that have been cut out from a video of a time period including the identified occurrence times.

Description

Information processing device, server device, and video display method

[0001] The present disclosure relates to a technology for allowing a user to watch videos and / or a technology for assisting a user in playing a game.

[0002] Videos of games played by various game players are uploaded to the video sharing server. When a user enters a string into the search box of a video sharing site, the video sharing server searches for game videos that match the entered string and provides the searched game videos to the user.

[0003] However, the game videos provided by video sharing servers do not always meet the user's expectations. Also, since users do not know where in the game video the scene of their interest is located, they have to watch the game video from the beginning to reach the scene of their interest.

[0004] When users want to watch game videos uploaded to a video sharing server, it is preferable that they be provided with game videos that include many scenes that interest them, so that they can watch the videos focusing on the scenes that they are interested in. Therefore, it is desirable to provide a system that can provide videos that meet users' wishes and enable users to watch videos efficiently.

[0005] It is usually difficult for users to evaluate their own play style. Therefore, it is desirable to analyze users' gameplay and objectively evaluate their play style. By presenting users with objective play style indicators, they can identify their lacking skills and increase their motivation to improve their skills.

[0006] Therefore, an object of the present disclosure is to provide a technology that allows users to watch videos efficiently, and a technology that supports users' gameplay by presenting an indicator of their playing style to the users.

[0007] An information processing device of one aspect of the present disclosure is an information processing device having a function for displaying videos, wherein the videos are set with metadata that associates information indicating actions occurring within the video with the times at which the actions occur, and the information processing device is equipped with a reception unit that receives input of a string indicating an action, an action identification unit that identifies multiple times at which actions related to the received string occur, and a display processing unit that displays a highlight video that connects multiple partial videos extracted from videos of a time period including the identified times at which the actions occur.

[0008] Another aspect of an information processing device of the present disclosure includes an execution unit that executes a game in accordance with user operations, a game video transmission unit that transmits a video of the executed game to a server device, an acquisition unit that acquires, from the server device, evaluation values ​​of actions by the user and comments regarding actions whose evaluation values ​​fall below a predetermined threshold, and a display processing unit that displays the acquired evaluation values ​​and comments.

[0009] A server device according to yet another aspect of the present disclosure is a server device that provides videos to a user, and includes a video recording unit that records videos with metadata that associates information indicating actions that occur within the video with the time the actions occurred, a search request receiving unit that acquires a search string from the user, a search unit that searches for a video in which an action related to the search string occurs relatively frequently from among multiple videos recorded in the video recording unit, and a video providing unit that provides the searched video to the user.

[0010] A server device according to yet another aspect of the present disclosure is a server device that assists a user in playing a game, and includes an analysis unit that derives the number of actions that occurred in each video from a video of a game played by the user and a video of a game played by multiple players other than the user; an evaluation unit that derives an evaluation value for an action by the user based on the number of actions taken by the multiple players and the number of actions taken by the user; and an information provision unit that, when the evaluation value for an action by the user falls below a predetermined threshold, provides the user with the evaluation value for the action by the user and a comment regarding the action whose evaluation value fell below the predetermined threshold.

[0011] A video display method according to yet another aspect of the present disclosure is a method for displaying a video on an information processing device, wherein the video has metadata set thereon that associates information indicating actions occurring within the video with the times at which the actions occur, and includes the steps of accepting input of a string of characters indicating the action, identifying multiple times at which the actions related to the accepted string of characters occur, and displaying a highlight video that connects multiple partial videos extracted from a video of a time period including the identified times at which the actions occur.

[0012] Any combination of the above components, and conversion of the present disclosure into a method, device, system, recording medium, computer program, etc., are also valid aspects of the present disclosure.

[0013] 1 is a diagram illustrating an information processing system according to an embodiment; FIG. 2 is a diagram illustrating a hardware configuration of an information processing device; FIG. 3 is a diagram illustrating an example of functional blocks of an information processing device; FIG. 4 is a diagram illustrating an example of a game screen; FIG. 5 is a diagram illustrating functional blocks of a video sharing server; FIG. 6 is a diagram illustrating an example of metadata extracted from image analysis results of a game video; FIG. 7 is a diagram illustrating an example of a home screen of a video sharing site; FIG. 8 is a diagram illustrating an example of a playback screen of a game video; FIG. 9 is a diagram illustrating an example of evaluation values ​​of actions by a user; FIG. 10 is a diagram illustrating an example of an average evaluation value; and FIG. 11 is a diagram illustrating an example of analysis results of a user's play style.

[0014] In the embodiment, an information processing device, which is a user terminal device, executes a game program. When an event occurs in the game, the game program outputs event information identifying the event to the system software. An event occurs when there is a change in the progress of game play or a change in the behavior of a player character controlled by a player. The game program incorporates multiple play units, each of which has at least a set end condition.

[0015] A game program includes activities, which are a type of unit of play and have set start and end conditions. Activities may be composed of stages, quests, missions, tournaments, sessions, etc. Activities are set appropriately by the game maker, and one quest that appears in the course of the game may constitute one activity, or multiple quests may constitute one activity. When multiple quests constitute one activity, completing all of the multiple quests is the end condition for that activity.

[0016] When an activity starts, the game program outputs to the system software an activity identifier (activity ID) that identifies the activity and event information including information indicating the start of the activity. When the activity ends, the game program outputs to the system software event information including the activity ID and information indicating the end of the activity. At the end of the activity, the game program may include the result of the activity (success or failure) in the event information. The system software generates event data by adding a user identifier (user account) that identifies the user, a game identifier (game ID) that identifies the game, and time information (timestamp) to the event information output from the game program, and transmits the event data to an external server device. The server device collects event data transmitted from multiple information processing devices operated by multiple players and analyzes play trends related to the activity. For example, the server device may statistically calculate the average play time of an activity based on the event data transmitted from the multiple information processing devices.

[0017] In the embodiment, the server device acquires video of a game played by multiple players and analyzes the game video. The server device may have an action recognition function that performs image analysis of the game video to detect actions of characters (player characters) operated by players. The server device may set the actions detected by the image analysis and the time at which the actions occurred as metadata for the game video, and use the metadata for image searches, etc. The server device may embed the metadata related to the actions in the game video, or may store the metadata in a database.

[0018] The server device of the embodiment provides a video sharing service that distributes game videos uploaded by multiple players to users. When a user enters a string of characters in a search box of a video sharing site, the video sharing server searches for game videos that match the entered string and provides the searched game videos to the user.

[0019] FIG. 1 illustrates an information processing system 1 according to an embodiment of the present disclosure. The information processing system 1 of the embodiment is a system in which a user can watch game videos, and includes an information processing device 10 operated by the user and a server device 5. An access point (hereinafter referred to as "AP") 8 has the functions of a wireless access point and a router, and the information processing device 10 connects to the AP 8 wirelessly or via a wired connection to be able to communicate with the server device 5 on a network 3. While FIG. 1 illustrates one user and one information processing device 10, in the information processing system 1, multiple information processing devices 10 operated by multiple users are connected to the server device 5 via the network 3.

[0020] The information processing device 10 is connected wirelessly or via a wire to an input device 6 operated by a user, and the input device 6 outputs information operated by the user to the information processing device 10. When the information processing device 10 receives operation information from the input device 6, it reflects the information in the processing of system software and game programs, and outputs the processing results from the output device 4. In the information processing system 1, the information processing device 10 is a game device (game console) that executes a game, and the input device 6 is a device such as a game controller that supplies user operation information to the information processing device 10. Note that the input device 6 may also be an input interface such as a keyboard or a mouse.

[0021] The auxiliary storage device 2 is a large-capacity storage device such as an HDD (hard disk drive) or an SSD (solid state drive), and may be a built-in storage device or an external storage device connected to the information processing device 10 via a USB (universal serial bus) or the like. The output device 4 may be a television having a display for outputting images and speakers for outputting sounds. The output device 4 may be connected to the information processing device 10 by a wired cable or wirelessly.

[0022] The camera 7, which is an imaging device, is provided near the output device 4 and captures images of the space around the output device 4. While Fig. 1 shows an example in which the camera 7 is attached to the top of the output device 4, it may be placed on the side or bottom of the output device 4, and in either case, it is placed in a position where it can capture an image of a user positioned in front of the output device 4. The camera 7 may be a stereo camera.

[0023] The server device 5 of the embodiment includes a management server 12 and a video sharing server 14. In the following description, the management server 12 and the video sharing server 14 are described as separate servers, but the management server 12 and the video sharing server 14 may be integrated into one server.

[0024] The management server 12 provides network services to users of the information processing system 1. The management server 12 manages user identifiers (user accounts) that identify each user, and each user signs in to the network services provided by the management server 12 using their user account. By signing in to the network service from their information processing device 10, users can register game save data and trophies, which are virtual rewards acquired during game play, on the management server 12. By registering save data and trophies on the management server 12, users can synchronize save data and trophies even if they use an information processing device other than the information processing device 10.

[0025] The management server 12 collects event data from multiple information processing devices 10 operated by multiple players (users). The management server 12 records the event data by linking it to user accounts. The management server 12 may perform statistical processing on the play times of activities from the collected event data to calculate the average play time of the activities, etc.

[0026] The video sharing server 14 provides a video sharing service that distributes video of games played by multiple players. The video sharing server 14 of the embodiment has a function of analyzing game videos uploaded by players and generating metadata for the game videos.

[0027] The video sharing server 14 performs image analysis on game videos acquired from multiple players to detect actions of player characters occurring in the game videos. The detected actions may be movements such as "punch," "kick," "guard," "jump," "run," and "jog." The video sharing server 14 sets the detected actions and the time of their occurrence as metadata for the game videos. The video sharing server 14 may embed the metadata related to the actions in the game videos, or may store it in a database. Note that the metadata related to the actions may be set by the management server 12 or another server, rather than by the video sharing server 14.

[0028] 2 shows the hardware configuration of the information processing device 10. The information processing device 10 includes a main power button 20, a power ON LED 21, a standby LED 22, a system controller 24, a clock 26, a device controller 30, a media drive 32, a USB module 34, a flash memory 36, a wireless communication module 38, a wired communication module 40, a subsystem 50, and a main system 60.

[0029] The main system 60 includes a main CPU (Central Processing Unit), a memory and memory controller serving as a main storage device, a GPU (Graphics Processing Unit), etc. The GPU is primarily used for processing game programs. The main CPU has the function of starting up system software and executing game programs installed in the auxiliary storage device 2 in the environment provided by the system software. The subsystem 50 includes a sub-CPU, a memory and memory controller serving as a main storage device, etc., but does not include a GPU.

[0030] While the main CPU has the function of executing a game program installed in the auxiliary storage device 2, the sub-CPU does not have such a function. However, the sub-CPU does have the function of accessing the auxiliary storage device 2 and the function of sending and receiving data to and from the server device 5. The sub-CPU is configured with only these limited processing functions, and therefore can operate with less power consumption than the main CPU. These functions of the sub-CPU are executed when the main CPU is in standby mode.

[0031] The main power button 20 is an input unit that receives operational input from the user, is provided on the front of the housing of the information processing device 10, and is operated to turn on or off the power supply to the main system 60 of the information processing device 10. The power ON LED 21 lights up when the main power button 20 is turned on, and the standby LED 22 lights up when the main power button 20 is turned off. The system controller 24 detects that the main power button 20 has been pressed by the user.

[0032] The clock 26 is a real-time clock that generates current date and time information and supplies it to the system controller 24 , the subsystem 50 , and the main system 60 .

[0033] The device controller 30 is configured as an LSI (Large-Scale Integrated Circuit) that, like a southbridge, transfers information between devices. As shown in the figure, devices such as the system controller 24, media drive 32, USB module 34, flash memory 36, wireless communication module 38, wired communication module 40, subsystem 50, and main system 60 are connected to the device controller 30. The device controller 30 absorbs differences in the electrical characteristics and data transfer speeds of each device and controls the timing of data transfer.

[0034] The media drive 32 is a drive device that is driven by loading a ROM medium 44 on which application software such as games and license information are recorded, and that reads programs, data, and the like from the ROM medium 44. The ROM medium 44 is a read-only recording medium such as an optical disk, a magneto-optical disk, or a Blu-ray disk.

[0035] The USB module 34 is a module that connects to external devices via a USB cable. The USB module 34 may also be connected to the auxiliary storage device 2 and the camera 7 via USB cables. The flash memory 36 is an auxiliary storage device that constitutes internal storage. The wireless communication module 38 wirelessly communicates with the input device 6 using a communication protocol such as Bluetooth (registered trademark) or IEEE802.11. The wired communication module 40 communicates with external devices via a wired connection and connects to the network 3 via the AP 8.

[0036] 3 shows an example of functional blocks of the information processing device 10. The information processing device 10 of the embodiment includes a processing unit 100 and a communication unit 102. The processing unit 100 includes a game execution unit 110, a game image generation unit 112, a display processing unit 114, an event information acquisition unit 120, an event data transmission unit 122, a game video acquisition unit 130, a game video transmission unit 132, an acquisition unit 140, a reception unit 142, and an action identification unit 144.

[0037] The functionality of the components in the information processing device 10 may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuits, and / or combinations thereof, configured or programmed to perform the functions described herein. A processor is considered to be circuitry or processing circuitry including transistors and other circuits. A processor may also be a programmed processor that executes a program stored in a memory.

[0038] In this specification, a circuit, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions.

[0039] If the hardware is a processor that is considered to be a type of circuitry, the circuitry, means, or unit may be a combination of hardware and software used to configure the hardware and / or processor.

[0040] The communication unit 102 has the functions of both the wireless communication module 38 and the wired communication module 40. While the user is playing a game, the communication unit 102 receives information (game operation information) relating to the user's operation of the input device 6, and provides the information to the game execution unit 110.

[0041] The game execution unit 110 executes a game in accordance with user operations. While the user is playing the game, the game execution unit 110 executes a game program based on game operation information and performs calculations to move a player character in a virtual space. The game image generation unit 112 includes a GPU (Graphics Processing Unit) and receives the results of calculations in the virtual space, generating game images from a viewpoint (virtual camera) in the virtual space. The game image generation unit 112 also generates game sounds at the viewpoint in the virtual space. The display processing unit 114 outputs the game images and game sounds generated by the game image generation unit 112 from the output device 4.

[0042] 4 shows an example of a game screen displayed on the output device 4. The user operates the input device 6 while viewing the game images and game sounds output from the output device 4 to progress through the game.

[0043] When an activity is started during the progress of a game, the game program outputs event information indicating the occurrence of an activity start event, and when the activity is ended, the game program outputs event information indicating the occurrence of an activity end event. When the event information acquisition unit 120 acquires the event information from the game program, it generates event data by adding to the event information a user account, a game ID, and time information (timestamp) indicating the time when the event occurred, and provides the event data to the event data transmission unit 122 and the game video acquisition unit 130. The game program may output the event information including the game ID and / or the timestamp to the event information acquisition unit 120.

[0044] A game developer may incorporate various activities into a game. For example, if a battle activity against an enemy boss is incorporated into a game, the game program outputs, at the start of the battle, event information including an activity ID that identifies the battle activity and information indicating the start of the battle activity. When the player wins the battle against the enemy boss, the game program outputs event information including an activity ID that identifies the battle activity, information indicating the end of the battle activity, and information indicating the success of the activity.

[0045] The event data transmission unit 122 transmits event data related to activities performed by players to the management server 12 via the communication unit 102. In the information processing system 1, the event data transmission process is performed by all information processing devices 10 connected to the management server 12, and the management server 12 acquires event data related to various activities of various games from the multiple information processing devices 10. Upon acquiring event information from a game program, the event information acquisition unit 120 preferably immediately generates event data to which a user account and a timestamp are added, and the event data transmission unit 122 transmits the event data to the management server 12.

[0046] The game video acquisition unit 130 has a ring buffer and has a function of automatically recording in the background the game video and game sounds generated by the game image generation unit 112 into the ring buffer. Hereinafter, for convenience of explanation, the game images and game sounds may also be simply referred to as "game video."

[0047] The ring buffer logically connects the beginning and end of a memory area to allow for cyclical use, and may be generated by setting the start and end addresses of the memory area in the auxiliary storage device 2. The start and end addresses of the ring buffer area may be set before shipping the information processing device 10, or may be freely set by the user. The game video acquisition unit 130 records game video data in the ring buffer in real time from the start address in a predetermined address order. Once the data reaches the end address, it returns to the start address and overwrites the data, repeating this process. For example, the ring buffer can record up to 60 minutes of game video. Therefore, the ring buffer will record game video played by the user over the last 60 minutes, along with time information (timestamps).

[0048] During game play, the user can generate user-generated content by extracting at least a portion of the game video recorded in the ring buffer. When the user operates a predetermined button on the input device 6 during game play, a content generation screen is displayed, and the user can select the time period for saving the most recent game play from multiple options (e.g., the most recent 30 seconds, the most recent minute, the most recent 5 minutes, etc.). The game video acquisition unit 130 extracts the game video for the most recent period selected by the user from the ring buffer and stores the extracted game video in the auxiliary storage device 2 as user-generated content.

[0049] As described above, the game video acquisition unit 130 is supplied with event information from the event information acquisition unit 120. The game video acquisition unit 130 records the event information along with the game video in the ring buffer. When cutting out a game video from the ring buffer, the game video acquisition unit 130 adds (embeds) information about events that occurred between the start time and end time of the game video to (as metadata for) the game video.

[0050] As another method for generating content, the user may specify a start point of the content in real time while playing a game, and then specify an end point of the content in real time. When the end point is specified, game video acquisition unit 130 extracts the game video from the ring buffer from the start point to the end point specified by the user, and stores the extracted game video in auxiliary storage device 2 as user-generated content. When extracting the game video from the start point to the end point, game video acquisition unit 130 adds event information that occurred between the start time and the end time of the game video to the game video as game video metadata.

[0051] In the embodiment, the game video acquisition unit 130 may have a function of automatically clipping game video recorded in the ring buffer. For example, when a battle activity against an enemy boss ends (when the player wins the battle against the enemy boss), the game video acquisition unit 130 may automatically read (cut out) game video from the ring buffer from the start time to the end time of the battle activity and record it as user-generated content in the auxiliary storage device 2. When clipping game video from the ring buffer, the game video acquisition unit 130 adds event information that occurred between the start time and the end time of the battle activity to the game video as game video metadata. In this way, game video (user-generated content) with event-related metadata added is accumulated in the auxiliary storage device 2.

[0052] A user can distribute user-generated content to an unspecified number of users by uploading the user-generated content to video sharing server 14. When a user operates input device 6 to input an instruction to upload user-generated content to information processing device 10, game video transmission unit 132 transmits the user-generated content (game video) recorded in auxiliary storage device 2 to video sharing server 14. Game video transmission unit 132 transmits the game video to which the user account and game ID have been added as metadata to video sharing server 14.

[0053] 5 shows functional blocks of the video sharing server 14 according to the embodiment. The video sharing server 14 includes a processing unit 200, a communication unit 202, and a recording device 240. The processing unit 200 includes a game video acquisition unit 210, an analysis unit 212, a metadata setting unit 214, a search request reception unit 216, a search unit 218, a video provision unit 220, an evaluation unit 222, and an information provision unit 224.

[0054] The functionality of the components in the video hosting server 14 may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuitry, and / or combinations thereof, configured or programmed to perform the functions described herein. A processor is considered to be circuitry or processing circuitry that includes transistors and other circuits. A processor may also be a programmed processor that executes a program stored in a memory.

[0055] In this specification, a circuit, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions.

[0056] If the hardware is a processor that is considered to be a type of circuitry, the circuitry, means, or unit may be a combination of hardware and software used to configure the hardware and / or processor.

[0057] The game video acquisition unit 210 acquires videos of games played by multiple players from multiple information processing devices 10, and records the videos in the video recording unit 242. The video recording unit 242 stores the game videos by linking them to the user identifiers (user accounts) of the players.

[0058] The analysis unit 212 has an AI (artificial intelligence) with an image analysis function, and performs image analysis of game videos using a trained model prepared for each game title. In this embodiment, the analysis unit 212 has a behavior recognition function that detects actions of a player character from game videos, and identifies the type of action (action name) and the time when the action occurred. The metadata setting unit 214 sets the time when the action occurred as metadata for the game video.

[0059] The trained model is an action detection model that has been trained through machine learning of game videos generated by multiple players. The action detection model performs machine learning using training game videos and information indicating actions performed by player characters in the game videos as training data. For the machine learning, deep learning techniques such as CNN, RNN, and LSTM may be used. When a game video is input, the trained action detection model is trained to output the name of the detected action and the time the action occurred (the time when the action was detected in the game video).

[0060] For example, an action detection model trained on game videos from the game title "Battle XZ" detects any of the following actions from the movements of the player character: "punch," "kick," "guard," "jump," "run," and "jog." Because the types of player character actions vary depending on the game title, it is preferable to generate an action detection model for each game title. Therefore, an action detection model trained on game videos from another game title may detect different types of actions from those in the game title "Battle XZ." Training an action detection model for each game title makes it possible to detect actions in game videos with high accuracy.

[0061] The metadata setting unit 214 sets the action names and action occurrence times extracted by image analysis of the game video as metadata for the game video. At this time, the metadata setting unit 214 may convert the action names into word vectors using a known conversion technique. These word vectors are generated using techniques called distributed representation or word embedding, and distributed representations of words with similar meanings are located close to each other in a vector space. In the example of the action in the game title "Battle XZ," the word vectors of "run" and "jog" are located close to each other in the vector space, so the cosine similarity between them is large. Therefore, "run" and "jog" can be considered to be similar actions.

[0062] FIG. 6A shows an example of metadata extracted from the image analysis results of a game video. In this example, the analysis unit 212 inputs video 1 of a game played by player A into the action detection model. Video 1 may be a video of player A playing activity X of the game title "Battle XZ." The analysis unit 212 provides the action names and their occurrence times output by the action detection model to the metadata setting unit 214. The analysis unit 212 also derives the number of times each action occurred and provides this to the metadata setting unit 214. The metadata setting unit 214 sets the occurrence time and the number of times each action occurred as metadata for video 1. In video 1, it has been detected that the player character controlled by player A punched eight times, kicked three times, guarded once, jumped once, ran twice, and jogged once. The metadata setting unit 214 records the metadata extracted for video 1 in the metadata recording unit 244. The metadata setting unit 214 may not only record the metadata extracted for the moving image 1 in the metadata recording unit 244 but also embed the metadata in the moving image 1 .

[0063] FIG. 6B shows another example of metadata extracted from the image analysis results of a game video. In this example, the analysis unit 212 inputs video 2 of a game played by player B into the action detection model. Video 2 may be a video of player B playing activity X of the game title "Battle XZ." The analysis unit 212 provides the action names and their occurrence times output by the action detection model to the metadata setting unit 214. The analysis unit 212 also derives the number of times each action occurred and provides this to the metadata setting unit 214. The metadata setting unit 214 sets the occurrence time of each action name and the number of times each action occurred as metadata for video 2. It has been detected that in video 2, the player character controlled by player B punched twice, kicked nine times, guarded zero times, jumped twice, ran once, and jogged zero times. The metadata setting unit 214 records the metadata extracted for video 2 in the metadata recording unit 244. The metadata setting unit 214 may not only record the metadata extracted for the moving image 2 in the metadata recording unit 244 but also embed the metadata in the moving image 2 .

[0064] 6(a) and 6(b), it is clear that player A tends to use punches frequently when defeating enemy characters, while player B tends to use kicks frequently when defeating enemy characters. Detecting actions included in game videos in this way makes it possible to estimate a player's playing style. In an embodiment, the metadata of game videos can be used for image search and can also be used to evaluate playing styles.

[0065] The metadata setting unit 214 sets the number of times each action occurs in a video as metadata, but may also set the number of times each action occurs per unit time as metadata. For example, if 10 punches are detected in each of two videos, video A and video B, and video A is 5 minutes long and video B is 1 minute long, punches are thrown more frequently in video B. Therefore, the analysis unit 212 may derive the number of times each action occurs per unit time (for example, 1 minute), and the metadata setting unit 214 may record the number of times each action occurs per unit time in the metadata recording unit 244.

[0066] The metadata setting unit 214 may identify a time period (action period) during which each action is concentrated, and record this action period as metadata in the metadata recording unit 244. The metadata setting unit 214 may identify action periods for all actions, or may identify action periods only for actions that are performed frequently. For example, the metadata setting unit 214 may identify the punch action period for video 1 as 0:00:15 to 0:00:41, and the kick action period for video 2 as 0:00:02 to 0:01:13.

[0067] <Provision of Video Sharing Service> As described above, the video sharing server 14 records a plurality of game videos uploaded by a plurality of players in the video recording unit 242, sets metadata for the plurality of game videos, and records the metadata in the metadata recording unit 244. In the embodiment, the game videos recorded in the video recording unit 242 are set with metadata that associates information indicating actions that occur in the videos (action names) with the times at which the actions occur.

[0068] A user can operate the information processing device 10 to access the video sharing server 14 and watch game videos uploaded to the video sharing server 14. The following describes the processing procedure when a user watches game videos recorded on the video sharing server 14.

[0069] 7 shows an example of a home screen of a video sharing site. A user inputs a string of characters representing a game video they wish to watch into a search box 300. In this example, the user inputs "game video of defeating an enemy with a kick" into the search box 300. When the user presses a search button 302, the information processing device 10 transmits a search request including the search string to the video sharing server 14.

[0070] In the video sharing server 14 , the search request receiving unit 216 receives a search request from the information processing device 10 , obtains a search character string included in the search request, and supplies the obtained character string to the search unit 218 .

[0071] The search unit 218 searches for videos that match the search string from among multiple videos recorded in the video recording unit 242. In this example, the search string is "game video of defeating an enemy with a kick," and the search unit 218 searches for videos in which the number of occurrences of an action related to the search string (here, kick) is relatively high.

[0072] At this time, the search unit 218 searches for videos in which an action related to the search string occurs relatively frequently, by referring to the metadata of the game videos recorded in the metadata recording unit 244. If metadata is embedded in the game videos, the search unit 218 searches for videos in which an action related to the search string occurs relatively frequently, by referring to the metadata embedded in each game video.

[0073] 6( a) and 6(b), since the number of times each action occurs is recorded as metadata in the metadata recording unit 244, the search unit 218 compares the number of times kicks occur in a plurality of game videos to identify a game video with a relatively large number of kicks. If the metadata includes the number of times an action occurs per unit time, the search unit 218 may compare the number of times kicks occur per unit time in a plurality of game videos to identify a game video with a relatively large number of kicks.

[0074] When the search unit 218 searches for game videos and identifies multiple candidate videos, the video providing unit 220 transmits thumbnail images of the multiple candidate videos to the information processing device 10. The information processing device 10 displays the multiple thumbnail images on the output device 4 in a selectable manner, and the user selects one of the thumbnail images. When the information processing device 10 notifies the video sharing server 14 of information indicating the thumbnail image selected by the user, the video providing unit 220 provides the game video selected by the user to the user's information processing device 10.

[0075] At this time, the video providing unit 220 provides the information processing device 10 with metadata of the game video along with the game video. The metadata may include information indicating time periods during which kicks occur relatively frequently. Note that if metadata is embedded in the game video, the video providing unit 220 does not need to provide the metadata to the information processing device 10 separately from the game video.

[0076] In the information processing device 10, the acquisition unit 140 acquires game videos and metadata from the video sharing server 14 and temporarily records them in a buffer memory (not shown). The display processing unit 114 reads out the game videos temporarily recorded in the memory and displays them on the output device 4. The display processing unit 114 has a function of efficiently playing back and displaying scenes desired by the user on the output device 4 by using the metadata of the game videos.

[0077] The following describes the case where a user selects video 2, whose metadata is shown in Fig. 6(b), as the video to watch. The metadata for video 2 also includes information indicating that the time period (action period) during which kicks occur relatively frequently is from 0:00:02 to 0:01:13.

[0078] 8 shows an example of a game video playback screen. The display processing unit 114 displays the game video acquired by the acquisition unit 140 from the video sharing server 14 in the first display area 310. The display processing unit 114 displays a seek bar 314 that represents the overall duration of the game video as the length of a bar and indicates the playback position of the game video as the position of a slider on the bar. In this example, the display processing unit 114 displays the seek bar 314 superimposed on the game video, but it may also be displayed outside the game video area.

[0079] A view button 316 for viewing the video of the action period is provided outside the game video area. When the user presses the view button 316, the display processing unit 114 plays back and displays the action period (0:00:02 to 0:01:13) in which kicks occur frequently. By being notified of the action period from the video sharing server 14 in this way, the user can view the game video of the period in which kicks occur frequently.

[0080] The display processing unit 114 displays information about actions included in the game video in a second display area 312, which is different from the first display area 310 that displays the game video. The second display area 312 is provided with a search box 320 for inputting a character string indicating an action, and the user can input the name of the action they want to view in the search box 320. In this example, the user inputs "kick" in the search box 320. When the user presses the search button 322, the receiving unit 142 receives the input of the character string indicating the action, and the action identification unit 144 identifies multiple occurrence times of the action related to the received character string.

[0081] When the action identification unit 144 recognizes that the received character string is "kick," it references the metadata of the game video and extracts the times at which kicks occurred in the game video. In this example, the game video provided by the video sharing server 14 is "video 2," and the acquisition unit 140 acquires the metadata shown in FIG. 6B . The action identification unit 144 then identifies the times at which kicks occurred in the game video as 0:00:02, 0:00:05, 0:00:15, 0:00:24, 0:00:31, 0:00:52, 0:01:05, 0:01:13, and 0:01:30. The action identification unit 144 notifies the display processing unit 114 of the identified kick times, and the display processing unit 114 displays the kick times as a time list 324 in the second display area 312.

[0082] A play button 330 for playing a video related to the action entered in the search box 320 is displayed above the time list 324. When the user presses the play button 330, the reception unit 142 receives an instruction to play the action video, and the display processing unit 114 extracts scenes including the entered action (here, a kick) from the game video and displays a highlight video made up of the extracted scenes on the output device 4.

[0083] Specifically, the display processing unit 114 cuts out a video of a time period including the time when the kick occurred, and connects the cut-out partial videos in chronological order of the time when the action occurred to display a highlight video that compiles kick scenes. For example, the partial videos may be video cut out from the time period from three hours (e.g., five seconds) before the time of occurrence to four hours (e.g., one second) after the time of occurrence. By connecting and displaying the partial videos, the display processing unit 114 allows the user to watch kick scenes included in the game video in succession. Note that when cutting out and connecting the partial videos, the display processing unit 114 preferably prevents adjacent partial videos from overlapping.

[0084] When the game video is streamed from the video sharing server 14, the action identification unit 144 transmits the identified multiple kick occurrence times to the video sharing server 14 from the communication unit 102. In the video sharing server 14, when the video providing unit 220 receives the multiple kick occurrence times, it cuts out a video of a time period including the kick occurrence time from video 2, and connects the cut-out multiple partial videos in chronological order of the action occurrence time to create a highlight video, which is streamed to the information processing device 10. In this way, the highlight video may be generated by either the information processing device 10 or the video sharing server 14.

[0085] As described above, according to the embodiment, a user can view a highlight video that compiles desired action scenes by inputting an action name in the search box 320. The display processing unit 114 or the video providing unit 220 has the function of dynamically generating a highlight video, which has the advantage that the video sharing server 14 does not need to create a highlight video in advance, and the user can view a highlight video that compiles his or her favorite action scenes.

[0086] In the embodiment, in the video sharing server 14, the search unit 218 searches for a video in which an action included in a search string occurs relatively frequently from among a plurality of videos recorded in the video recording unit 242. At this time, the search unit 218 may search for a video in which an action related to the search string occurs relatively frequently by using the number of occurrences obtained by adding the number of occurrences of an action that matches the action included in the search string and the number of occurrences of an action similar to the action included in the search string.

[0087] To determine the similarity of actions, the search unit 218 uses word vectors of the action names to determine whether the two actions are similar. When it is determined from the word vectors that "run" and "jog" are similar as described above, the search unit 218 may search for a video with a relatively high number of occurrences from among the multiple videos recorded in the video recording unit 242, using the number of occurrences obtained by adding the number of occurrences of "run" and "jog" when the search string includes "run."

[0088] In the embodiment, the action identification unit 144 in the information processing device 10 identifies multiple occurrence times of actions that match the action indicated by the character string input into the search box 320. In addition to the occurrence times of actions that match the action indicated by the character string, the action identification unit 144 may also identify occurrence times of actions that are similar to the action indicated by the character string.

[0089] <Evaluation of Playing Style> As described above, the video sharing server 14 records multiple game videos uploaded by multiple players in the video recording unit 242, sets metadata for the multiple game videos, and records the metadata in the metadata recording unit 244. Below, we will explain a method for evaluating a user's playing style using the metadata of the game videos stored in the video sharing server 14. Hereinafter, the user whose playing style is to be evaluated will be referred to as "User C."

[0090] In the combat activity of the game title "Battle XZ," the player character can perform the following actions: "punch," "kick," "guard," "jump," "run," and "jog." The analysis unit 212 performs image analysis of a video of the game played by the player to detect actions included in the game video. The analysis unit 212 derives the number of times each action has occurred from the action detection results, and the metadata setting unit 214 sets the time of occurrence of each action and the number of times each action has occurred as metadata for the game video (see FIGS. 6( a) and 6(b)).

[0091] When evaluating the play style, the analysis unit 212 derives the number of actions that occurred in the game video from the video of the activity played by user C. The number of actions that occurred in activity A played by user C is shown below: Punch: 2 times Kick: 15 times Guard: 1 time Jump: 2 times Run: 3 times Jog: 1 time

[0092] The analysis unit 212 derives the number of actions that occurred in a game video from a video of the same activity A played by multiple players other than user C. For example, the analysis unit 212 may derive the number of actions taken by each player from a video of activity A played by 1,000 or more players.

[0093] The evaluation unit 222 derives an evaluation value for an action by user C based on the number of times an action is performed by multiple players and the number of times an action is performed by user C. As described above, if 10 punches are detected in each of two videos A and B, and if video A is 5 minutes long and video B is 1 minute long, punches are thrown more frequently in video B. Therefore, the analysis unit 212 may derive the number of times each action is performed per unit time (for example, 1 minute), and the evaluation unit 222 may derive an evaluation value for an action by user C based on the number of times an action is performed per unit time by multiple players and the number of times an action is performed per unit time by user C.

[0094] The evaluation unit 222 may derive the evaluation value of the action by user C as a normalized score, for example, by scaling (normalizing) the minimum value to 0 and the maximum value to 10. As another derivation method, the evaluation unit 222 may derive the evaluation value of the action by user C by standardization processing using the deviation from the average value of the number of occurrences. As yet another derivation method, the evaluation unit 222 may derive the evaluation value of the action by user C as a deviation value.

[0095] FIG. 9 shows an example of evaluation values ​​for actions by user C. In the example shown in FIG. 9, the evaluation unit 222 derives the evaluation value for each action as a normalized score. A normalized score of 5 indicates the average number of times the action is performed, and the greater the normalized score is than 5, the more frequently the action is performed, and the smaller the normalized score is than 5, the less frequently the action is performed. By the evaluation unit 222 deriving evaluation values ​​for actions by user C for each type of action, it is possible to objectively evaluate user C's playing style. In this example, it can be seen that in user C's play of activity A, the number of kicks is very high, while the number of punches is significantly low.

[0096] The evaluation unit 222 derives an evaluation value of an action by user C for each activity and calculates an average evaluation value by averaging the evaluation values ​​derived for multiple activities. Fig. 10 shows an example of an average evaluation value obtained by averaging the evaluation values ​​for multiple activities. For example, the evaluation unit 222 may derive an average evaluation value for each action by averaging the evaluation values ​​(e.g., normalized scores) derived for several tens of activities.

[0097] The information providing unit 224 provides the average evaluation value of each activity to the information processing device 10 of user C. When the average evaluation value of actions by user C is below a predetermined threshold, the information providing unit 224 may provide the information processing device 10 of user C with comments related to the actions whose average evaluation value is below the predetermined threshold, along with the average evaluation value of each activity. For example, when the average evaluation value is below the threshold (2.5), the information providing unit 224 provides the information processing device 10 of user C with comments related to the activity. The comments may be automatically generated using a large-scale language model (LLM). Note that the information providing unit 224 may also provide the information processing device 10 of user C with the average evaluation value of each action by multiple players for comparison.

[0098] In the information processing device 10, the acquisition unit 140 acquires, from the video sharing server 14, information indicating the evaluation value of each action by user C, information indicating the evaluation values ​​of each action by multiple players, and comments on actions for which the evaluation value by user C is below a predetermined threshold. The display processing unit 114 displays the acquired evaluation values ​​and comments on the output device 4.

[0099] FIG. 11 is an example screen showing the analysis results of user C's play style. The display processing unit 114 displays, on the output device 4, the evaluation value of each action of user C and comments regarding "punches" for which the evaluation value fell below a threshold. The display processing unit 114 may display the action evaluation values ​​of user C and all players in the form of a radar chart. Displaying the evaluation values ​​in radar chart format allows user C to recognize at a glance which actions he is good at and which actions he is not good at. Furthermore, by viewing the comments, the user can be more motivated to improve his punching skills.

[0100] In this example, the information providing unit 224 presents activity XY, which cannot be completed without throwing punches at a high frequency, to user C as practice material. By playing activity XY, user C can practice throwing punches. The information providing unit 224 also provides user C with a game video of player A defeating an enemy primarily using punches. In this way, by referencing the metadata of the game video, the information providing unit 224 can identify activity XY, which cannot be completed without throwing punches at a high frequency, and can also identify a game video of player A defeating an enemy primarily using punches.

[0101] The present disclosure has been described above based on examples. These examples are merely illustrative, and it will be understood by those skilled in the art that various modifications are possible in the combination of the components and processing steps, and that such modifications are also within the scope of the present disclosure.

[0102] In a modified example, the information processing system 1 may include a cloud game server (not shown) that executes a game program in accordance with user operations. In this modified example, the information processing device 10 transmits information (game operation information) about user operations on the input device 6 to the game server. The game server generates game images and game sounds based on the game operation information and streams the game images and game sounds to the information processing device 10. In this modified example, the information processing device 10 does not need to have a function for executing a game program, and may be a terminal device that outputs game images and game sounds from the output device 4.

[0103] The present disclosure may include the following aspects. [Item 1] An information processing device having a function to display moving images, wherein the moving images are set with metadata associating information indicating actions occurring in the moving images with times at which the actions occurred, and the information processing device includes a circuit configured to: accept input of a string indicating an action; identify multiple times at which the action related to the accepted string occurred; and display a highlight moving image by connecting multiple partial moving images extracted from a time period including the identified times at which the action occurred. [Item 2] The information processing device according to item 1, wherein the circuit displays multiple times at which the action related to the accepted string occurred in a second display area different from a first display area for displaying the moving images. [Item 3] The information processing device according to item 1, wherein the circuit identifies times at which an action similar to the action indicated by the accepted string occurred, in addition to times at which an action matched the action indicated by the accepted string occurred. [Item 4] The information processing device according to item 1, wherein the circuit displays a highlight moving image by connecting multiple partial moving images in chronological order of the time at which the action occurred. [Item 5] An information processing device comprising a circuit configured to: execute a game in accordance with a user's operation; transmit a video of the executed game to a server device; obtain, from the server device, evaluation values ​​of actions by the user and comments regarding actions whose evaluation values ​​are below a predetermined threshold; and display the obtained evaluation values ​​and comments.[Item 6] A server device that provides videos to a user, the server device comprising: a video recording device that records videos with metadata that associates information indicating actions that occur in the videos with the times when the actions occurred; and a circuit configured to: obtain a search string from the user; search for videos from multiple videos recorded on the video recording device that have a relatively high frequency of actions related to the search string; and provide the searched videos to the user. [Item 7] The server device according to Item 6, wherein the circuit provides the user with information indicating time periods when the action occurs relatively frequently, along with the videos. [Item 8] The server device according to Item 6, wherein the circuit searches for videos that have a relatively high frequency of actions related to the search string, using a frequency obtained by adding together the number of occurrences of actions that match the actions included in the search string and the number of occurrences of actions similar to the actions included in the search string. [Item 9] A server device that assists a user in playing a game, comprising: a circuit configured to: derive the number of actions that occurred in each video of a game played by the user and videos of a game played by multiple players other than the user; derive an evaluation value of an action by the user based on the number of actions occurred by the multiple players and the number of actions occurred by the user; and if the evaluation value of an action by the user is below a predetermined threshold, provide the user with the evaluation value of the action by the user and a comment regarding the action whose evaluation value is below the predetermined threshold. [Item 10] The server device according to item 9, wherein the circuit derives the number of actions per unit time. [Item 11] The server device according to item 9, wherein the circuit derives the evaluation value of an action by the user for each unit of play in the game.[Item 12] The server device according to item 11, wherein the circuit derives an evaluation value of an action by a user for each unit of play, calculates an average evaluation value by averaging the evaluation values ​​for each unit of play, and provides the user with a comment regarding the action if the average evaluation value of the user's action is below a predetermined threshold. [Item 12] The server device according to item 9, wherein the circuit derives the number of times the action occurs for each type of action, and derives an evaluation value of the user's action for each type of action. [Item 14] A method for displaying videos on an information processing device, wherein the video has metadata set that associates information indicating actions that occur within the video with times at which the actions occurred, accepts input of a string of characters indicating the actions, identifies multiple times at which actions related to the accepted string of characters occur, and displays a highlight video by connecting multiple partial videos extracted from video of a time period including the identified times at which the actions occurred.

[0104] The present disclosure can be used in the technical fields of allowing users to watch videos and / or assisting users in playing games.

[0105] 1: information processing system, 2: auxiliary storage device, 3: network, 4: output device, 5: server device, 6: input device, 10: information processing device, 12: management server, 14: video sharing server, 100: processing unit, 102: communication unit, 110: game execution unit, 112: game image generation unit, 114: display processing unit, 120: event information acquisition unit, 122: event data transmission unit, 130: game video acquisition unit, 132: game video transmission unit, 140: acquisition unit, 142: reception unit, 144: action identification unit, 200: processing unit, 202: communication unit, 210: game video acquisition unit, 212: analysis unit, 214: metadata setting unit, 216: search request reception unit, 218: search unit, 220: video provision unit, 222: evaluation unit, 224: information provision unit, 240: recording device, 242: video recording unit, 244: metadata recording unit.

Claims

1. An information processing device having a function to display videos, wherein the videos have metadata set that associates information indicating actions that occur within the video with the times at which the actions occur, and the information processing device is characterized by comprising: a reception unit that receives input of a string indicating an action; an action identification unit that identifies multiple times at which actions related to the received string occur; and a display processing unit that displays a highlight video that connects multiple partial videos extracted from a video of a time period that includes the identified times at which the actions occur.

2. The information processing device according to claim 1, characterized in that the display processing unit displays multiple occurrence times of actions related to the received character string in a second display area different from the first display area that displays the video.

3. The information processing device according to claim 1, characterized in that the action identification unit identifies the occurrence time of an action similar to the action indicated by the received character string in addition to the occurrence time of an action that matches the action indicated by the received character string.

4. The information processing device according to claim 1, wherein the display processing section displays a highlight video in which a plurality of partial videos are joined together in chronological order of the time when the action occurred.

5. An information processing device comprising: an execution unit that executes a game in accordance with user operations; a game video transmission unit that transmits a video of the executed game to a server device; an acquisition unit that acquires, from the server device, evaluation values ​​of actions by users and comments regarding actions whose evaluation values ​​are below a predetermined threshold; and a display processing unit that displays the acquired evaluation values ​​and comments.

6. A server device that provides videos to a user, comprising: a video recording unit that records videos with metadata that associates information indicating actions that occur in the video with the time the actions occurred; a search request receiving unit that acquires a search string from the user; a search unit that searches for a video that has a relatively high number of occurrences of actions related to the search string from among multiple videos recorded in the video recording unit; and a video providing unit that provides the searched video to the user.

7. The server device according to claim 6, wherein the video providing unit provides the user with information indicating time periods when the frequency of occurrence of an action is relatively high, along with the video.

8. The server device according to claim 6, characterized in that the search unit searches for videos in which actions related to the search string occur relatively frequently, using a count obtained by adding together the number of occurrences of actions that match the actions included in the search string and the number of occurrences of actions similar to the actions included in the search string.

9. A server device that supports a user's game play, comprising: an analysis unit that derives the number of actions that occurred in each video from a video of a game played by the user and a video of a game played by multiple players other than the user; an evaluation unit that derives an evaluation value of an action by the user based on the number of actions occurred by multiple players and the number of actions occurred by the user; and an information provision unit that, when the evaluation value of an action by the user is below a predetermined threshold, provides the user with the evaluation value of the action by the user and a comment regarding the action whose evaluation value is below the predetermined threshold.

10. The server device according to claim 9, wherein the analysis unit derives the number of actions per unit time.

11. The server device according to claim 9, wherein the evaluation unit derives an evaluation value of an action by a user for each unit of play in a game.

12. The server device described in claim 11, characterized in that the evaluation unit derives an evaluation value for an action by a user for each unit of play and calculates an average evaluation value by averaging the evaluation values ​​for each unit of play, and the information providing unit provides the user with a comment regarding the action when the average evaluation value for the action by the user is below a predetermined threshold.

13. The server device according to claim 9, wherein the analysis unit derives the number of times an action occurs for each type of action, and the evaluation unit derives an evaluation value of the action by the user for each type of action.

14. A method for displaying moving images on an information processing device, wherein the moving image has metadata set that associates information indicating actions occurring within the moving image with the times at which the actions occur, the method comprising the steps of: accepting input of a string indicating the action; identifying multiple times at which the action related to the accepted string occurs; and displaying a highlight moving image by connecting multiple partial moving images extracted from a moving image of a time period including the identified times at which the action occurs.

Images