Information processing device and highlight moving image generation method

Abstract

An event identification unit 144 identifies a plurality of events having occurred in a moving image. An evaluation value derivation unit 146 derives an evaluation value of the identified plurality of events. An editing processing unit 148 cuts out an event scene including the time at which an event occurred, in a time length based on the evaluation value of the event, and generates a highlight moving image connecting two or more event scenes.

Description

Information processing device and highlight video generating method

[0001] The present disclosure relates to a technique for automatically generating a highlight video.

[0002] Patent Document 1 discloses a technology that calculates an evaluation value for all event scenes included in multiple game videos over a predetermined period of time, and automatically creates a highlight video by connecting event scenes whose evaluation value exceeds a predetermined threshold.

[0003] Japanese Patent Application Laid-Open No. 2022-21632

[0004] Video sharing sites provide an environment where users can post videos and view videos posted by others. In recent years, short videos have become increasingly popular due to their ease of viewing. As a result, in order to have many users watch videos, it has become necessary to shorten the length of the videos. However, editing using video editing tools requires skill, and editing can be very tedious for users without editing skills. Therefore, an object of the present disclosure is to provide a technology for automatically generating highlight videos.

[0005] An information processing device according to one aspect of the present disclosure includes an event identification unit that identifies multiple events that occur within a video, an evaluation value derivation unit that derives evaluation values ​​for the multiple identified events, and an editing processing unit that cuts out an event scene that includes the time at which the event occurred with a duration based on the evaluation value of the event, and generates a highlight video by connecting two or more event scenes.

[0006] A highlight video generation method according to another aspect of the present disclosure includes the steps of identifying a plurality of events that occurred within a video, deriving evaluation values ​​for the identified plurality of events, extracting an event scene including the time at which the event occurred with a duration based on the evaluation value of the event, and generating a highlight video by connecting two or more event scenes.

[0007] Any combination of the above components, and conversion of the present disclosure into a method, device, system, recording medium, computer program, etc., are also valid aspects of the present disclosure.

[0008] 1 is a diagram illustrating an information processing system according to an embodiment; FIG. 2 is a diagram illustrating a hardware configuration of an information processing device; FIG. 3 is a diagram illustrating an example of functional blocks of an information processing device; FIG. 4 is a diagram illustrating an example of a game screen; FIG. 5 is a diagram illustrating a flowchart of a process for generating a highlight video; FIG. 6 is a diagram illustrating examples of letters and numbers extracted from a game image; FIG. 7 is a diagram illustrating an example of a plurality of identified events; FIG. 8 is a diagram illustrating evaluation values; FIG. 9 is a diagram illustrating an example of an event selection screen; and FIG.

[0009] In the embodiment, an information processing device, which is a user terminal device, executes a game program. When an event occurs in the game, the game program outputs event information identifying the event to the system software. An event occurs when there is a change in the progress of game play, a change in the behavior of a character (player character) controlled by a player, or the like.

[0010] A game is provided with activities, each of which has a start condition and an end condition. An activity may be made up of stages, quests, tournaments, sessions, etc. Activities are set appropriately by the game maker, and one quest that appears in the course of the game may constitute one activity, or multiple quests may constitute one activity. When multiple quests constitute one activity, completing all of the multiple quests is the end condition for that activity.

[0011] When an activity starts, the game program outputs to the system software an activity identifier (activity ID) that identifies the activity and event information including information indicating the start of the activity. When the activity ends, the game program outputs to the system software event information including the activity ID and information indicating the end of the activity. At the end of the activity, the game program may include the result of the activity (success or failure) in the event information. The system software generates event data by adding a user identifier (user account) that identifies the user, a game identifier (game ID) that identifies the game, and time information (timestamp) to the event information output from the game program, and transmits the event data to an external server device. The server device collects event data transmitted from multiple information processing devices operated by multiple players and analyzes play trends related to the activity. For example, the server device may statistically calculate the average play time of an activity based on the event data transmitted from the multiple information processing devices.

[0012] In order to increase motivation for playing the game, various missions are set in the game, and when a user completes a mission, the user is awarded a virtual award (trophy) corresponding to the mission. Multiple trophies are available in the game according to the difficulty of the mission, with bronze, silver, gold, and platinum trophies available in order of difficulty.

[0013] The conditions for unlocking a trophy (i.e., the conditions for a user to complete a mission and obtain a trophy) may be associated with an activity, but may also be set independently of the activity. If a mission for a bronze trophy, such as "defeat a mini-boss," is set, when a user defeats the mini-boss, the game program outputs event information indicating that the bronze trophy has been unlocked, and the user is awarded the bronze trophy. If a mission for a silver trophy, such as "defeat a mid-boss," is set, when a user defeats the mid-boss, the game program outputs event information indicating that the silver trophy has been unlocked, and the user is awarded the silver trophy. If a mission for a gold trophy, such as "defeat a final boss," is set, when a user defeats the final boss, the game program outputs event information indicating that the gold trophy has been unlocked, and the user is awarded the gold trophy.

[0014] FIG. 1 shows an information processing system 1 according to an embodiment. The information processing system 1 of the embodiment includes an information processing device 10 operated by a user and a server device 5. An access point (hereinafter referred to as "AP") 8 has the functions of a wireless access point and a router, and the information processing device 10 connects to the AP 8 wirelessly or via a wired connection to be able to communicate with the server device 5 on a network 3. While FIG. 1 shows one user and one information processing device 10, in the information processing system 1, multiple information processing devices 10 operated by multiple users are connected to the server device 5 via the network 3.

[0015] The information processing device 10 is connected wirelessly or via a wire to an input device 6 operated by a user, and the input device 6 outputs information operated by the user to the information processing device 10. When the information processing device 10 receives operation information from the input device 6, it reflects the information in the processing of system software and game programs, and outputs the processing results from the output device 4. In the information processing system 1, the information processing device 10 is a game device (game console) that executes a game, and the input device 6 is a device such as a game controller that supplies user operation information to the information processing device 10. Note that the input device 6 may also be an input interface such as a keyboard or a mouse.

[0016] The auxiliary storage device 2 is a large-capacity storage device such as an HDD (hard disk drive) or an SSD (solid state drive), and may be a built-in storage device or an external storage device connected to the information processing device 10 via a USB (universal serial bus) or the like. The output device 4 may be a television having a display for outputting images and speakers for outputting sounds. The output device 4 may be connected to the information processing device 10 by a wired cable or wirelessly.

[0017] The camera 7, which is an imaging device, is provided near the output device 4 and captures images of the space around the output device 4. While Fig. 1 shows an example in which the camera 7 is attached to the top of the output device 4, it may be placed on the side or bottom of the output device 4, and in either case, it is placed in a position where it can capture an image of a user positioned in front of the output device 4. The camera 7 may be a stereo camera.

[0018] The server device 5 provides network services to users of the information processing system 1. The server device 5 manages user identifiers (user accounts) that identify each user, and each user signs in to the network services provided by the server device 5 using their user account. By signing in to the network service from their information processing device 10, users can register game save data and trophies, which are virtual rewards acquired during game play, on the server device 5. By registering save data and trophies on the server device 5, users can synchronize save data and trophies even if they use an information processing device other than the information processing device 10.

[0019] The server device 5 collects event data from multiple information processing devices 10 operated by multiple players (users). The server device 5 records the event data by linking it to user accounts. The server device 5 may perform statistical processing of the play time of an activity from the collected event data to calculate the average play time of the activity, etc. The server device 5 may also derive what percentage of the play time of each player ranks in the top among all players.

[0020] 2 shows the hardware configuration of the information processing device 10. The information processing device 10 includes a main power button 20, a power ON LED 21, a standby LED 22, a system controller 24, a clock 26, a device controller 30, a media drive 32, a USB module 34, a flash memory 36, a wireless communication module 38, a wired communication module 40, a subsystem 50, and a main system 60.

[0021] The main system 60 includes a main CPU (Central Processing Unit), a memory and memory controller serving as a main storage device, a GPU (Graphics Processing Unit), etc. The GPU is primarily used for processing game programs. The main CPU has the function of starting up system software and executing game programs installed in the auxiliary storage device 2 in the environment provided by the system software. The subsystem 50 includes a sub-CPU, a memory and memory controller serving as a main storage device, etc., but does not include a GPU.

[0022] While the main CPU has the function of executing a game program installed in the auxiliary storage device 2, the sub-CPU does not have such a function. However, the sub-CPU does have the function of accessing the auxiliary storage device 2 and the function of sending and receiving data to and from the server device 5. The sub-CPU is configured with only these limited processing functions, and therefore can operate with less power consumption than the main CPU. These functions of the sub-CPU are executed when the main CPU is in standby mode.

[0023] The main power button 20 is an input unit that receives operational input from the user, is provided on the front of the housing of the information processing device 10, and is operated to turn on or off the power supply to the main system 60 of the information processing device 10. The power ON LED 21 lights up when the main power button 20 is turned on, and the standby LED 22 lights up when the main power button 20 is turned off. The system controller 24 detects that the main power button 20 has been pressed by the user.

[0024] The clock 26 is a real-time clock that generates current date and time information and supplies it to the system controller 24 , the subsystem 50 , and the main system 60 .

[0025] The device controller 30 is configured as an LSI (Large-Scale Integrated Circuit) that, like a southbridge, transfers information between devices. As shown in the figure, devices such as the system controller 24, media drive 32, USB module 34, flash memory 36, wireless communication module 38, wired communication module 40, subsystem 50, and main system 60 are connected to the device controller 30. The device controller 30 absorbs differences in the electrical characteristics and data transfer speeds of each device and controls the timing of data transfer.

[0026] The media drive 32 is a drive device that is driven by loading a ROM medium 44 on which application software such as games and license information are recorded, and that reads programs, data, and the like from the ROM medium 44. The ROM medium 44 is a read-only recording medium such as an optical disk, a magneto-optical disk, or a Blu-ray disk.

[0027] The USB module 34 is a module that connects to external devices via a USB cable. The USB module 34 may also be connected to the auxiliary storage device 2 and the camera 7 via USB cables. The flash memory 36 is an auxiliary storage device that constitutes internal storage. The wireless communication module 38 wirelessly communicates with the input device 6 using a communication protocol such as Bluetooth (registered trademark) or IEEE802.11. The wired communication module 40 communicates with external devices via a wired connection and connects to the network 3 via the AP 8.

[0028] 3 shows an example of functional blocks of the information processing device 10. The information processing device 10 of the embodiment includes a processing unit 100 and a communication unit 102. The processing unit 100 includes a game execution unit 110, a game image generation unit 112, a display processing unit 114, an event information acquisition unit 120, an event data transmission unit 122, a game video acquisition unit 130, a reception unit 140, an image analysis unit 142, an event identification unit 144, an evaluation value derivation unit 146, and an editing processing unit 148. The storage device 150 includes an event data storage unit 152, a game video storage unit 154, and a highlight video storage unit 156. The storage device 150 may be the auxiliary storage device 2.

[0029] The functionality of the components in the information processing device 10 may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuits, and / or combinations thereof, configured or programmed to perform the functions described herein. A processor is considered to be circuitry or processing circuitry including transistors and other circuits. A processor may also be a programmed processor that executes a program stored in a memory.

[0030] In this specification, a circuit, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions.

[0031] If the hardware is a processor that is considered to be a type of circuitry, the circuitry, means, or unit may be a combination of hardware and software used to configure the hardware and / or processor.

[0032] The communication unit 102 has the functions of both the wireless communication module 38 and the wired communication module 40. While the user is playing a game, the communication unit 102 receives information (game operation information) relating to the user's operation of the input device 6, and provides the information to the game execution unit 110.

[0033] The game execution unit 110 executes a game in accordance with user operations. While the user is playing the game, the game execution unit 110 executes a game program based on game operation information and performs calculations to move a player character in a virtual space. The game image generation unit 112 includes a GPU (Graphics Processing Unit) and receives the results of calculations in the virtual space to generate game images from a viewpoint (virtual camera) in the virtual space. The game image generation unit 112 also generates game sounds at the viewpoint in the virtual space. The display processing unit 114 outputs the game images and game sounds generated by the game image generation unit 112 from the output device 4.

[0034] 4 shows an example of a game screen displayed on the output device 4. The user operates the input device 6 while viewing the game screen and game sounds output from the output device 4 to progress through the game.

[0035] When an event occurs during the progress of a game, the game program outputs event information indicating the occurrence of the event. When the event information acquisition unit 120 acquires the event information from the game program, it generates event data by adding to the event information a user account, a game ID, and time information (timestamp) indicating the time the event occurred. The event information acquisition unit 120 records the event data in the event data storage unit 152 and provides it to the event data transmission unit 122.

[0036] A game developer may incorporate various events into a game. For example, if a battle activity against an enemy boss is incorporated into a game, the game program will output, at the start of the battle, event information including an activity ID that identifies the battle activity and information indicating the start of the battle activity. When the player wins the battle against the enemy boss, the game program will output event information including an activity ID that identifies the battle activity, information indicating the end of the battle activity, and information indicating the success of the activity. Also, as mentioned above, when the unlock condition for a trophy is met, the game program will output event information indicating that the trophy has been unlocked.

[0037] The event data transmission unit 122 transmits the event data to the server device 5 via the communication unit 102. In the information processing system 1, the event data transmission process is performed by all information processing devices 10 connected to the server device 5, and the server device 5 acquires event data related to various activities of various games from the multiple information processing devices 10. When event information is output from the game program, the event data transmission unit 122 preferably transmits the event data to the server device 5 immediately.

[0038] In the embodiment, the game video acquisition unit 130 has a ring buffer and has a function of automatically recording, in the background, the game video and game sounds generated by the game image generation unit 112 into the ring buffer. Hereinafter, for convenience of explanation, the game images and game sounds may also be simply referred to as "game video."

[0039] The ring buffer logically connects the beginning and end of a memory area to allow for cyclical use, and may be generated by setting the start and end addresses of the memory area in the auxiliary storage device 2. The start and end addresses of the ring buffer area may be set before shipping the information processing device 10, or may be freely set by the user. The game video acquisition unit 130 records game video data in the ring buffer in real time from the start address in a predetermined address order. Once the data reaches the end address, it returns to the start address and overwrites the data, repeating this process. For example, the ring buffer can record up to 60 minutes of game video. Therefore, the ring buffer will record game video played by the user over the last 60 minutes, along with time information (timestamps).

[0040] During game play, a user can generate user-generated content by extracting at least a portion of the game video recorded in the ring buffer. When the user operates a predetermined button on input device 6 during game play, a content generation screen is superimposed on the game screen, and the user can select a time period for saving the most recent game play from multiple options (e.g., the last 30 seconds, the last minute, the last 5 minutes, the last 15 minutes, the last 60 minutes, etc.). When the user selects the saving time, game video acquisition unit 130 extracts game video for the time period selected by the user from the ring buffer and stores the extracted game video in game video storage unit 154 as user-generated content. At this time, game video acquisition unit 130 adds the start time and end time of the user-generated content to the user-generated content as metadata.

[0041] As another method for generating content, the user may specify a start point of the content in real time while playing a game, and then specify an end point of the content in real time. Once the end point is specified, game video acquisition unit 130 extracts the game video from the ring buffer from the start point to the end point specified by the user, and stores the extracted game video as user-generated content in game video storage unit 154. At this time, game video acquisition unit 130 adds the start time and end time of the user-generated content to the user-generated content as metadata.

[0042] In this way, the user can save the user-generated content in the game video storage unit 154. The processing unit 100 of the embodiment has a function of automatically generating a highlight video from the user-generated content.

[0043] 5 shows a flowchart of the process of generating a highlight video from user-generated content. When receiving unit 140 receives from a user an instruction to generate a highlight video specifying user-generated content, the highlight video generation process begins. Note that the process of generating a highlight video of the user-generated content may be automatically started when the user-generated content is saved in game video storage unit 154.

[0044] First, the event identification unit 144 identifies multiple events that have occurred within the user-generated content (S10). As will be described below, the event identification unit 144 uses multiple means to identify events within the user-generated content. Then, the evaluation value derivation unit 146 derives evaluation values ​​for the multiple identified events (S12). The editing processing unit 148 extracts an event scene that includes the time at which the event occurred, with a duration based on the evaluation value of the event, and generates a highlight video by connecting two or more event scenes (S14). Each step is described in detail below.

[0045] <S10: Event Identification Process> When an event occurs within the game, the game program of the embodiment outputs event information that identifies the event that has occurred, and the output event information is stored in event data storage unit 152. Event identification unit 144 reads from event data storage unit 152 the event information that has been output from the game program between the start time and end time of user-generated content (hereinafter sometimes simply referred to as "content"), and identifies the event that has occurred within the content.

[0046] In addition to the event information output from the game program, the event identification unit 144 has a function of identifying events that have occurred within the content from the analysis results of the image analysis unit 142. In this embodiment, the image analysis unit 142 includes AI (artificial intelligence) with an image analysis function. The image analysis unit 142 uses a trained model to perform image analysis of the content and extract things (events) that the user has achieved within the content.

[0047] The image analysis unit 142 has an optical character recognition (OCR) function and recognizes text included in the content to extract character data. For example, referring to the game screen shown in Figure 4, letters and numbers are displayed in the upper left corner of the screen, and the image analysis unit 142 extracts the letters and numbers included in the game screen using the character recognition function.

[0048] FIG. 6 shows examples of letters and numbers extracted from a game image by the image analysis unit 142. The image analysis unit 142 sets a coordinate system for the game image, with the horizontal axis being the X axis and the vertical axis being the Y axis, and identifies the coordinates of the recognized letters and numbers. In the example shown in FIG. 6, the image analysis unit 142 extracts the number "150" at the position (x1, y3), the letters "HP" at the position (x1, y4), the letters "COMBO" at the position (x2, y1), and the numbers "121" at the position (x2, y2). The image analysis unit 142 may derive the center of gravity of the letter or number as the coordinate of the letter or number.

[0049] The image analysis unit 142 identifies the object indicated by the number based on the relative positions of the letters and numbers. In this example, "HP" is displayed near the number displayed at the (x1, y3) position, and "combo" is displayed near the number displayed at the (x2, y2) position. Therefore, the image analysis unit 142 identifies that the number displayed at the (x1, y3) position represents "HP (hit points)" and the number displayed at the (x2, y2) position represents "combo number." The image analysis unit 142 notifies the event identification unit 144 that the player character's HP at this point is 150 and the combo number is 121. In this way, the image analysis unit 142 notifies the event identification unit 144 of the letters and numbers extracted from the content along with time information.

[0050] The event identification unit 144 may monitor the numbers displayed at the position (x2, y2) between the start time and the end time of the content, and identify the largest number. For example, if the largest number is "143," the event identification unit 144 may determine that the largest combo number in the content is "143," and identify the achievement of the combo number "143" as an event.

[0051] Furthermore, the event identification unit 144 may monitor the number displayed at the position (x1, y3) between the start time and the end time of the content to determine whether or not the HP has changed. If the HP displayed at the position (x1, y3) remains at 150 and the player character defeats all enemy characters, the event identification unit 144 determines that the enemy characters have been defeated without taking any damage. In this case, the event identification unit 144 may identify the defeat of the enemy characters without taking any damage as an event. On the other hand, if the HP falls below 150, the event identification unit 144 determines that the player character has taken damage, and does not detect an event in which the enemy characters have been defeated without taking any damage.

[0052] The image analysis unit 142 also has an action recognition function and detects the actions of the player character in the content. The image analysis unit 142 may use a recurrent neural network (RNN). For example, in a fighting game, the image analysis unit 142 detects the actions of the player character and notifies the event identification unit 144 of the detected player character actions together with time information. The image analysis unit 142 may identify actions such as a "sword attack" or a "kick" of the player character. The image analysis unit 142 notifies the event identification unit 144 of the player character actions detected from the content together with time information.

[0053] The event identification unit 144 may count the actions detected between the start time and the end time of the content and identify the number of times each action has been performed. For example, if the event identification unit 144 identifies that 50 sword attacks have been performed in the content, it may identify the 50 sword attacks as an event.

[0054] The image analysis unit 142 may have a VLM (Vision and Language Model) and detect the actions of the player character in the content. The image analysis unit 142 uses the VLM to convert the actions of the player character in the content into text and notify the event identification unit 144 of the converted actions of the player character along with time information. Instead of or in addition to the above-described action recognition function, the image analysis unit 142 creates a text description of the actions of the player character in the content and notifies the event identification unit 144 of the text. For example, the image analysis unit 142 may generate text from the content stating, "The enemy boss was defeated with a sword attack," and notify the event identification unit 144 of the text stating, "The enemy boss was defeated with a sword attack." The event identification unit 144 may identify, from the text stating, "The enemy boss was defeated with a sword attack," that the enemy boss was defeated with a sword attack as an event.

[0055] As described above, the event identification unit 144 identifies a plurality of events that have occurred within the content from the event data stored in the event data storage unit 152 and / or the analysis results from the image analysis unit 142 .

[0056] 7 shows an example of multiple events identified by the event identification unit 144. In FIG. 7, the horizontal axis represents the time from the start time to the end time of the content, with the upper row showing events detected by the game program and the lower row showing events detected by image analysis. The event identification unit 144 of the embodiment expresses each event in text as shown in FIG. 7.

[0057] In the example shown in Figure 7, the following events are detected by the game program: (GP1) Bronze trophy "Obtained a sword" is acquired (GP2) Silver trophy "125 combo" is acquired (GP3) Gold trophy "Defeat enemy boss" is acquired (GP4) Quest "Defeat 100 enemies" is completed

[0058] Furthermore, the following events were detected through image analysis: (ML1) Achieving 143 combos (ML2) Defeating an enemy boss with a sword attack (ML3) 50 sword attacks

[0059] The event identification unit 144 checks whether an event detected by the game program and an event detected by image analysis overlap. In the example shown in Figure 7, the event shown in the upper row and the event shown in the lower row were detected by different means, so there is a possibility that the event shown in the upper row and the event shown in the lower row are the same event. Therefore, the event identification unit 144 determines whether the two events are the same event, and if it determines that they are the same event, it is necessary to treat the two events as a single event.

[0060] The event identification unit 144 may determine whether two events are the same event based on the text representing the detected events. The event identification unit 144 converts the text representing the events into a sentence vector using a known conversion technique. This sentence vector is generated using a technique called distributed representation or word embedding, and distributed representations of words or sentences with similar meanings are located close to each other in a vector space. For example, the event identification unit 144 may vectorize the text representing the events using a model called Sentence-Transformers.

[0061] The similarity between the sentence vectors is evaluated using, for example, cosine similarity. The event identification unit 144 may calculate the cosine similarity between the sentence vectors of two events and determine that the two events are the same if the cosine similarity exceeds a predetermined threshold.

[0062] 7, the event identification unit 144 determines that (GP2) and (ML1) indicate the same event because the cosine similarity exceeds a predetermined threshold. In other words, the event identification unit 144 determines that (GP2) Obtaining a silver trophy "125 combos" and (ML1) Achieving 143 combos are the same event, and handles these two events as a single event.

[0063] Furthermore, since the cosine similarity exceeds a predetermined threshold, the event identification unit 144 determines that (GP3) and (ML2) indicate the same event. In other words, the event identification unit 144 determines that (GP3) Obtaining the gold trophy "defeating the enemy boss" and (ML2) Defeating the enemy boss with a sword attack are the same event, and treats these two events as a single event.

[0064] The event identification unit 144 may determine whether two events are the same event based on the occurrence times of the detected events. Specifically, the event identification unit 144 may determine that two events are the same event if the occurrence times of the two events are the same or if the difference between the occurrence times is within a predetermined time.

[0065] 7 , the event identification unit 144 determines that (GP2) and (ML1) are the same event by confirming that the time difference between the time t2 when (GP2) occurred and the time t3 when (ML1) occurred is within a predetermined time. In addition, the event identification unit 144 determines that (GP3) and (ML2) are the same event by confirming that the time t4 when (GP3) occurred is the same as the time t4 when (ML2) occurred.

[0066] In addition, the event identification unit 144 may determine that two events are the same event if the cosine similarity between the sentence vectors of the two events exceeds a predetermined threshold and the difference in the occurrence times of the two events is within a predetermined time period.

[0067] <S12: Evaluation Value Deriving Process> When the event identifying unit 144 identifies a plurality of events that have occurred in the content, the evaluation value deriving unit 146 derives evaluation values ​​for the plurality of events.

[0068] (Evaluation value for taking trophies) In the information processing system 1 of the embodiment, a base point is assigned in advance to each type of trophy. The evaluation value deriving unit 146 derives an evaluation value for taking trophies using the base point and the percentage of trophies taken. The percentage of trophies taken is the percentage of the number of players who have taken trophies, and is calculated statistically in the server device 5. The evaluation value deriving unit 146 obtains the percentage of trophies taken from the server device 5.

[0069] Here, let us assume that the base points for a gold trophy are 1, the base points for a silver trophy are 0.5, and the base points for a bronze trophy are 0.1. Also, let us assume that the trophy acquisition rate for the bronze trophy "Obtained a sword" is 0.5, the silver trophy "125 combo" is 0.3, and the gold trophy "Defeat enemy boss" is 0.2. In this case, the evaluation value derivation unit 146 derives the evaluation value for trophy acquisition as follows: (Evaluation value for trophy acquisition) = trophy base points x (1 - acquisition rate)

[0070] Therefore, the evaluation values ​​for winning trophies in GP1 to GP3 are calculated as follows: (GP1 evaluation value) 0.1 x (1 - 0.5) = 0.05 (GP2 evaluation value) 0.5 x (1 - 0.3) = 0.35 (GP3 evaluation value) 1.0 x (1 - 0.2) = 0.8

[0071] (Evaluation Value for Quest Completion) In the information processing system 1 of the embodiment, a base point is assigned to each quest, and the evaluation value derivation unit 146 derives an evaluation value for quest completion using the base point, the completion rate, and information on the user's completion speed. Here, the completion rate is the percentage of the number of players who have completed the quest, and the information on the user's completion speed is information indicating what percentage of all players the time taken for the user to complete the quest is in, and each is calculated statistically in the server device 5. The evaluation value derivation unit 146 derives the evaluation value for quest completion as follows: (Evaluation Value for Quest Completion) = Quest Base Points × (1 - Completion Rate) × (1 - Top % / 100)

[0072] Here, let's assume that the base score for the quest "Defeat 100 enemies" is 0.5, the completion rate is 0.8, and the user's completion time is in the top 22%. Therefore, the evaluation value for completing the GP4 quest is calculated as follows: (GP4 evaluation value) 0.5 × (1 - 0.8) × (1 - 22 / 100) = 0.078.

[0073] (Evaluation Value for Number of Combos) When deriving the evaluation value for the number of combos, the evaluation value derivation unit 146 derives the evaluation value based on information indicating what percentage of the user's combo count is in the top among all players. Here, the information indicating what percentage is in the top is statistically determined in the server device 5 by each information processing device 10 notifying the server device 5 of the highest number of combos. The evaluation value derivation unit 146 derives the evaluation value for the number of combos as follows: (Evaluation Value for Number of Combos) = 1 - Top % / 100

[0074] Here, if the combo number 143 is ranked in the top 50%, the evaluation value for the combo number of ML1 is calculated as follows: (evaluation value of ML1) 1-50 / 100=0.5.

[0075] (Evaluation value for defeating an enemy boss) When deriving an evaluation value for defeating an enemy boss, the evaluation value derivation unit 146 derives the evaluation value based on the proportion of the number of players who have defeated the enemy boss (defeat rate) among all players. Here, the proportion of the number of players who have defeated the enemy boss is determined statistically in the server device 5. The evaluation value derivation unit 146 derives the evaluation value for defeating the enemy boss as follows: (Evaluation value for defeating an enemy boss) = 1 - defeat rate

[0076] Here, if the defeat rate is 0.25, the evaluation value for defeating the enemy boss of ML2 is calculated as follows: (evaluation value of ML2) 1 - 0.25 = 0.75.

[0077] (Evaluation Value for Number of Sword Attacks) When deriving the evaluation value for the number of sword attacks, the evaluation value derivation unit 146 derives the evaluation value based on information indicating what percentage of the user's number of sword attacks falls in the top among all players. Here, the information indicating what percentage of the user's number of sword attacks falls in the top is statistically determined in the server device 5 by each information processing device 10 notifying the server device 5 of the highest number of sword attacks. The evaluation value derivation unit 146 derives the evaluation value for the number of sword attacks as follows: (Evaluation Value for Number of Sword Attacks) = 1 - Top % / 100

[0078] Here, if 50 sword attacks is ranked in the top 80%, the evaluation value for the number of sword attacks of ML3 is calculated as follows: (evaluation value of ML3) 1-80 / 100=0.2.

[0079] As described above, the evaluation value derivation unit 146 derives evaluation values ​​for the events GP1 to GP4 and ML1 to ML3. In this embodiment, the event identification unit 144 determines that GP2 and ML1 are the same event, and that GP3 ​​and ML2 are the same event. Therefore, the evaluation value derivation unit 146 needs to unify the evaluation values ​​of an event obtained by merging two events.

[0080] Here, the evaluation value of GP2 is 0.35, and the evaluation value of ML1 is 0.5. The evaluation value derivation unit 146 may adopt the higher evaluation value, and therefore set the evaluation values ​​of GP2 and ML1 to 0.5. The evaluation value derivation unit 146 may represent the two events with the event having the higher evaluation value, and therefore represent GP2 and ML1 with ML1.

[0081] The evaluation value of GP3 is 0.8, and the evaluation value of ML2 is 0.75. Therefore, the evaluation value derivation unit 146 sets the evaluation values ​​of GP3 and ML2 to 0.8, and represents GP3 and ML2. Figure 8 shows the evaluation values ​​of each event derived by the evaluation value derivation unit 146.

[0082] <S14: Highlight Video Generation Process> When the evaluation value derivation unit 146 derives evaluation values ​​for multiple events, the editing processing unit 148 generates an event selection screen, and the display processing unit 114 displays the event selection screen on the output device 4. Fig. 9 shows an example of the event selection screen. The display processing unit 114 displays the event selection screen in which events are arranged from top to bottom in descending order of evaluation value so that they can be selected. In this embodiment, a highlight video can include up to three event scenes, and the user can select up to three events. Note that a highlight video may also include four or more event scenes.

[0083] In the embodiment, the event identification unit 144 identifies five events, and therefore the five events are displayed as selectable events on the event selection screen. However, it is also possible that the event identification unit 144 identifies a significantly larger number of events. In such a case, it is not desirable to display all events on the event selection screen, thereby unnecessarily increasing the number of options. Therefore, it is preferable that the editing processing unit 148 presents the user with a finite number of events with relatively high evaluation values ​​as selectable events. For example, the editing processing unit 148 may include up to 10 events on the event selection screen.

[0084] When arranging the events from top to bottom in descending order of evaluation value, the event identification unit 144 may pre-select the top three events in evaluation value. This eliminates the need for the user to select the top three events. If the user wants to include another event in the highlight video, the user can deselect the selected event and then select another event.

[0085] When the user selects two or more events on the event selection screen and operates the input device 6 to press the generate button 170, the editing processing unit 148 extracts an event scene including the time when the event occurred with a duration based on the evaluation value of the event, and generates a highlight video of a predetermined duration by connecting the two or more event scenes. The duration of the highlight video is, for example, 15 seconds, and the editing processing unit 148 creates a 15-second short video by connecting the two or more event scenes.

[0086] In the example shown in Fig. 9, the following events are selected: (GP3) Obtain the gold trophy "defeat enemy boss" (ML1) Achieve 143 combo (ML3) 50 sword attacks The editing processing unit 148 determines the duration of each event scene from the evaluation value of each event.

[0087] 8 , the evaluation values ​​for each event are as follows: (GP3) 0.8 (ML1) 0.5 (ML3) 0.2 For example, the editing processing unit 148 may determine the duration of each event scene so that the ratio of the evaluation values ​​matches the ratio of the durations. Because the duration of the highlight video is 15 seconds, the editing processing unit 148 determines the duration of each event scene as follows: (GP3) 8 seconds (ML1) 5 seconds (ML3) 2 seconds

[0088] 10 shows an example of a highlight video. The editing processing unit 148 extracts an event scene including the time when an event occurred, with a duration based on the evaluation value of the event, and generates a highlight video by connecting multiple event scenes. Specifically, the editing processing unit 148 extracts an 8-second event scene including both the time t4 when GP3 occurred and the time t3 when ML1 occurred, extracts a 5-second event scene including both the time t3 when ML1 occurred and the time t6 when ML3 occurred, and generates a highlight video by connecting each of these event scenes.

[0089] For example, the editing processing unit 148 may extract each event scene from the user-generated content so that the end point is a predetermined time (e.g., 0.5 seconds) after the occurrence time. In this case, the editing processing unit 148 extracts, as the event scene for GP3, an 8-second video that starts 7.5 seconds before the occurrence time t4 of GP3 and ends 0.5 seconds after the occurrence time t4. The editing processing unit 148 also extracts, as the event scene for ML1, a 5-second video that starts 4.5 seconds before the occurrence time t3 of ML1 and ends 0.5 seconds after the occurrence time t3, and as the event scene for ML3, a 5-second video that starts 1.5 seconds before the occurrence time t6 of ML3 and ends 0.5 seconds after the occurrence time t6. The editing processing unit 148 connects the three extracted event scenes to generate a highlight video.

[0090] In the embodiment, the editing processing unit 148 connects the extracted event scenes in descending order of evaluation value, but the editing processing unit 148 may connect the extracted event scenes in descending order of evaluation value.Furthermore, the editing processing unit 148 may connect the extracted event scenes in ascending order of occurrence time.

[0091] In the embodiment, the editing processing unit 148 determines the duration of each event scene so that the ratio of the evaluation values ​​matches the ratio of the durations, but the duration of the event scenes may also be determined, for example, in order of the evaluation values. For example, the event scene with the highest evaluation value may be set to 7 seconds, the event scene with the second highest evaluation value to 5 seconds, and the event scene with the third highest evaluation value to 3 seconds. Even in this case, it is preferable that the higher the evaluation value, the longer the scene duration.

[0092] The editing processing unit 148 stores the generated highlight video in the highlight video storage unit 156. The user may upload the highlight video stored in the highlight video storage unit 156 to the video sharing server at any timing. Furthermore, the editing processing unit 148 may automatically generate the highlight video and then automatically upload the highlight video to the video sharing server.

[0093] According to the embodiment, a highlight video can be automatically generated without the user having to perform editing work on the content. The user can select events to be included in the highlight video on the event selection screen, so that the user's preferences can be reflected in the highlight video.

[0094] The present disclosure has been described above based on examples. These examples are merely illustrative, and it will be understood by those skilled in the art that various modifications are possible in the combination of the components and processing steps, and that such modifications are also within the scope of the present disclosure.

[0095] In a modified example, the information processing system 1 may include a cloud game server (not shown) that executes a game program in accordance with user operations. In this modified example, the information processing device 10 transmits information (game operation information) about user operations on the input device 6 to the game server. The game server generates game images and game sounds based on the game operation information and streams the game images and game sounds to the information processing device 10. In this modified example, the information processing device 10 does not need to have a function for executing a game program, and may be a terminal device that outputs game images and game sounds from the output device 4. In this modified example, the cloud game server or another server may operate as the information processing device 10 described in the embodiment and have a function for automatically generating a highlight video.

[0096] The present disclosure may include the following aspects. [Item 1] An information processing device including a circuit configured to: identify multiple events that occurred in a video; derive evaluation values ​​for the identified multiple events; extract event scenes including times when the events occurred with durations based on the evaluation values ​​of the events; and generate a highlight video by connecting two or more event scenes. [Item 2] The information processing device according to item 1, wherein the circuit creates a highlight video of a predetermined duration. [Item 3] The information processing device according to item 2, wherein the circuit determines the duration of each event scene from the evaluation value of each event. [Item 4] The information processing device according to item 1, wherein the circuit creates a highlight video by connecting event scenes of two or more events selected by a user. [Item 5] The information processing device according to item 4, wherein the circuit presents events with relatively high evaluation values ​​to the user so that they can be selected. [Item 6] The information processing device according to item 1, wherein the circuit determines whether the two identified events are the same event, and if determined to be the same event, treats the two events as a single event. [Item 7] The information processing device according to item 6, wherein the circuit determines whether the two events are the same event based on text indicating the event that has occurred. [Item 8] The information processing device according to item 6, wherein the circuit determines whether the two events are the same event based on the time the event occurred. [Item 9] A method for generating a highlight video in an information processing device, comprising: identifying a plurality of events that have occurred within a video; deriving evaluation values ​​for the identified plurality of events; extracting event scenes including times at which the events occurred, with a duration based on the evaluation values ​​of the events; and generating a highlight video by connecting two or more event scenes.[Item 10] A recording medium having recorded thereon a program executed on a computer of an information processing device, the program causing the computer to realize the following functions: identifying a plurality of events that occurred within a video; deriving evaluation values ​​for the identified plurality of events; extracting an event scene including the time at which the event occurred with a duration based on the evaluation value of the event; and generating a highlight video by connecting two or more event scenes.

[0097] The present disclosure can be applied to the technical field of generating highlight videos from user-generated content.

[0098] DESCRIPTION OF SYMBOLS 1: Information processing system, 2: Auxiliary storage device, 3: Network, 4: Output device, 5: Server device, 6: Input device, 7: Camera, 8: AP, 10: Information processing device, 100: Processing unit, 102: Communication unit, 110: Game execution unit, 112: Game image generation unit, 114: Display processing unit, 120: Event information acquisition unit, 122: Event data transmission unit, 130: Game video acquisition unit, 140: Reception unit, 142: Image analysis unit, 144: Event identification unit, 146: Evaluation value derivation unit, 148: Editing processing unit, 150: Storage device, 152: Event data storage unit, 154: Game video storage unit, 156: Highlight video storage unit.

Claims

1. An information processing device comprising: an event identification unit that identifies multiple events that occur within a video; an evaluation value derivation unit that derives evaluation values ​​for the multiple identified events; and an editing processing unit that cuts out event scenes that include the time at which the events occurred with a duration based on the evaluation value of the events, and generates a highlight video by connecting two or more event scenes.

2. The information processing device according to claim 1, wherein the editing processing section creates a highlight video of a predetermined length of time.

3. The information processing device according to claim 2, wherein the editing processing unit determines the duration of each event scene from the evaluation value of each event.

4. The information processing device according to claim 1, wherein the editing processing section creates a highlight video by connecting event scenes of two or more events selected by the user.

5. The information processing device according to claim 4, wherein the editing processing unit presents to the user events with relatively high evaluation values ​​so that the events can be selected.

6. The information processing device according to claim 1, characterized in that the event identification unit determines whether the two identified events are the same event, and if it determines that they are the same event, treats the two events as a single event.

7. The information processing device according to claim 6, wherein the event identification unit determines whether two events are the same event based on text indicating the event that has occurred.

8. The information processing device according to claim 6, wherein the event identification unit determines whether two events are the same event based on the occurrence times of the events.

9. A method for generating a highlight video, comprising the steps of: identifying a plurality of events that occurred within a video; deriving evaluation values ​​for the identified plurality of events; extracting an event scene including the time at which the event occurred with a duration based on the evaluation value of the event; and generating a highlight video by connecting two or more event scenes.

10. A program for enabling a computer to perform the following functions: identify multiple events that occur within a video; derive evaluation values ​​for the identified multiple events; extract an event scene containing the time at which the event occurred with a duration based on the evaluation value of the event; and generate a highlight video by connecting two or more event scenes.

Images