Information processing system and game image sharing method

Abstract

A game image acquisition unit 210 acquires an image in a game played by a user in the past or an image in a game being played by the user. A content identification unit 212 performs image analysis on the acquired game image to determine whether or not the game image is related to content shared with another user in the past. A sharing processing part 214 causes the game image, which has been determined to be related to the shared content in the past, to be shared with another user.

Description

Information processing system and game image sharing method

[0001] The present disclosure relates to a technology for saving images of a game that a user is currently playing or has played in the past as content and / or a technology for sharing the content with other users.

[0002] Patent Literature 1 discloses an information processing device that executes a game program based on user operations, displays game images on an output device, and records the game images in a ring buffer in the background. The user specifies the start and end points for cutting out game video recorded in the ring buffer on an editing screen, and uploads the cut-out game video to a content sharing server.

[0003] Japanese Patent Application Laid-Open No. 2020-870

[0004] Conventionally, users manually capture images of the game they are currently playing to generate original content. Generally, the content generated by a user is called user-generated content, and the user shares the user-generated content with other users by uploading the user-generated content to a shared server or providing it to specific friends.

[0005] An object of the present disclosure is to provide a mechanism for efficiently realizing the process of generating content and the process of sharing content with other users.

[0006] An information processing system according to one aspect of the present disclosure includes a game image acquisition unit that acquires images of games that a user has played in the past or images of games that the user is currently playing, a content identification unit that performs image analysis on the acquired game images to determine whether the game images are related to content that the user has shared with other users in the past, and a sharing processing unit that allows game images that have been determined to be related to past shared content to be shared with other users.

[0007] Another aspect of the present disclosure is a game image sharing method including steps of acquiring an image of a game that a user has played in the past or an image of a game that the user is currently playing, performing image analysis on the acquired game image to determine whether the acquired game image is related to content that the user has shared with another user in the past, and allowing the game image determined to be related to the previously shared content to be shared with another user.

[0008] Any combination of the above components, and conversion of the present disclosure into a method, device, system, recording medium, computer program, etc., are also valid aspects of the present disclosure.

[0009] 1 is a diagram illustrating an information processing system according to an embodiment; FIG. 2 is a diagram illustrating a hardware configuration of an information processing device; FIG. 3 is a diagram illustrating an example of functional blocks of an information processing device; FIG. 4 is a diagram illustrating an example of a game image to be displayed; FIG. 5 is a flowchart illustrating automatic generation of content; FIG. 6 is a diagram illustrating an example of a game image to be displayed; FIG. 7 is a diagram illustrating a method for cutting out a game moving image; FIG. 8 is a diagram illustrating another example of functional blocks of an information processing device; FIG. 9 is a flowchart illustrating content sharing processing; FIG. 10 is a diagram illustrating an example of a screen on which content is presented; FIG. 11 is a diagram illustrating another example of a screen on which content is presented.

[0010] In an information processing system according to an embodiment, an information processing device may execute a game program based on user operations, output game images and game sounds to an output device, and automatically generate user-original content including game images and / or game sounds without user instructions. The automatically generated content may be still images (screenshots) of the game or video images of the game. The information processing device may also have a function for automatically sharing content with other users without user instructions. The content sharing process may include providing the content to other users (e.g., friends) or uploading the content to a distribution server so that other users can view it. The information processing device may automatically perform a sharing process to determine a sharing destination and provide the content to the sharing destination, or may present the content and the sharing destination to the user before performing the sharing process and obtain the user's consent before performing the sharing process.

[0011] 1 illustrates an information processing system 1 according to an embodiment of the present disclosure. The information processing system 1 of the embodiment includes an information processing device 10 operated by a user. The information processing system 1 may include a management server 5 that provides network services to users and a content sharing server 9 that distributes content. The information processing system 1 may also include another server, such as a cloud game server.

[0012] The access point (hereinafter referred to as "AP") 8 has the functions of a wireless access point and a router, and the information processing device 10 connects to the AP 8 wirelessly or via a wired connection to be able to communicate with the management server 5 and content sharing server 9 on a network 3 such as the Internet.

[0013] The information processing device 10 is connected wirelessly or by wire to an input device 6 having a plurality of operation members, and the input device 6 outputs information of the user's operation of the operation members to the information processing device 10. When the information processing device 10 receives operation information from the input device 6, it reflects the information in the processing of system software and game software, and causes the output device 4 to output the processing results. In the information processing system 1, the information processing device 10 is a game device (game console) that executes a game, and the input device 6 is an operation device such as a game controller that supplies information of the user's operation of the operation members to the information processing device 10. Note that the input device 6 may also be an input interface such as a keyboard or a mouse.

[0014] In a modified example, the information processing system 1 may include a cloud game server (not shown) that executes a game program based on user operations. In this modified example, the information processing device 10 transmits information (game operation information) about user operations on the input device 6 to the game server. The game server generates game images and game sounds based on the game operation information and streams the game images and game sounds to the information processing device 10. In this modified example, the information processing device 10 does not need to have a function for executing a game program, and may be a terminal device that outputs game images and game sounds from the output device 4.

[0015] The auxiliary storage device 2 is a large-capacity storage device such as an HDD (hard disk drive) or an SSD (solid state drive), and may be a built-in storage device or an external storage device connected to the information processing device 10 via a USB (universal serial bus) or the like. The output device 4 may be a television having a display for outputting images and speakers for outputting sounds. The output device 4 may be connected to the information processing device 10 by a wired cable or wirelessly.

[0016] The camera 7, which is an imaging device, is provided near the output device 4 and captures images of the space around the output device 4. While Fig. 1 shows an example in which the camera 7 is attached to the top of the output device 4, it may be placed on the side or bottom of the output device 4, and in either case, it is placed in a position where it can capture an image of a user positioned in front of the output device 4. The camera 7 may be a stereo camera.

[0017] The management server 5 provides network services to users of the information processing system 1. The management server 5 manages user accounts that identify each user, and each user signs in to the network services provided by the management server 5 using their user account. By signing in to the network service from their information processing device 10, users can register game save data and trophies, which are virtual rewards acquired during game play, on the management server 5. By registering save data and trophies on the management server 5, users can synchronize save data and trophies even when using an information processing device other than the information processing device 10. User accounts are associated with account information such as paid membership of a specific network service.

[0018] The content sharing server 9 makes the content generated by the information processing device 10 available for viewing by specific or unspecified users. Users participating in the information processing system 1 can view desired content by accessing the content sharing server 9 and selecting the content. In the embodiment, the information processing device 10 automatically identifies content suitable for sharing with other users from among multiple pieces of content stored therein and uploads the content to the content sharing server 9. In this case, the information processing device 10 may automatically upload the content to the content sharing server 9, or may upload the content to the content sharing server 9 only upon obtaining the user's consent. Note that the process of uploading content to the content sharing server 9 is one form of sharing content with other users, and the information processing device 10 may also share the content with a specific user by sending the content to the specific user (e.g., a friend).

[0019] The information processing device 10 includes a content generation model that has learned from content that users have manually generated in the past (hereinafter also referred to as "user-generated content"). The content generation model performs machine learning on multiple pieces of user-generated content to extract patterns (features) from the user-generated content. User-generated content is generated according to the preferences of each user, and each user has different characteristics. The more user-generated content is used for machine learning, the more accurately features that accurately capture the patterns in the user-generated content can be extracted, thereby increasing the reliability of the content generation model.

[0020] The content generation model may be trained to input an image of a game currently being played by a user, and output a score value representing the degree to which the game image reflects the characteristics of the user-generated content (similarity). The similarity indicates the degree to which the game image matches the user-generated content, and the score value may be expressed on a scale of 100 points.

[0021] The content generation model (machine learning model) may be configured as a convolutional neural network (CNN) that corresponds to a multi-layer neural network including an input layer, one or more convolutional layers, and an output layer. The information processing device 10 may train each coupling coefficient (weight) in the CNN using a learning method such as deep learning. Note that the information processing device 10 may train the content generation model using other AI training methods.

[0022] The information processing device 10 also includes a shared content identification model that learns content that a user has previously shared with other users (hereinafter also referred to as "shared content"). The shared content identification model performs machine learning on multiple pieces of shared content to extract patterns (features) of the shared content. Shared content is content that a user selects from among user-generated content that the user wishes to share with other users, and each user has different characteristics. The more shared content is used for machine learning, the more accurately features that accurately capture the patterns of the shared content can be extracted, thereby increasing the reliability of the shared content identification model.

[0023] The shared content identification model may be trained to input content, which is an image of a game played by a user in the past, and output a score value representing the degree to which the content reflects the characteristics of the shared content (similarity). The similarity indicates the degree to which the content matches the shared content, and the score value may be expressed on a scale of 100 points.

[0024] The shared content identification model (machine learning model) may be configured as a convolutional neural network (CNN) that corresponds to a multi-layer neural network including an input layer, one or more convolutional layers, and an output layer. The information processing device 10 may train each coupling coefficient (weight) in the CNN using a learning method such as deep learning. Note that the information processing device 10 may train the shared content identification model using another AI training method.

[0025] 2 shows the hardware configuration of the information processing device 10. The information processing device 10 includes a main power button 20, a power ON LED 21, a standby LED 22, a system controller 24, a clock 26, a device controller 30, a media drive 32, a USB module 34, a flash memory 36, a wireless communication module 38, a wired communication module 40, a subsystem 50, and a main system 60.

[0026] The main system 60 includes a main CPU (Central Processing Unit), a memory and memory controller serving as a main storage device, a GPU (Graphics Processing Unit), etc. The GPU is primarily used for processing game programs. The main CPU has the function of starting up system software and executing game programs installed in the auxiliary storage device 2 in the environment provided by the system software. The subsystem 50 includes a sub-CPU, a memory and memory controller serving as a main storage device, etc., but does not include a GPU.

[0027] While the main CPU has the function of executing game programs installed in the auxiliary storage device 2, the sub-CPU does not have such a function. However, the sub-CPU does have the function of accessing the auxiliary storage device 2 and the function of sending and receiving data to and from the management server 5. The sub-CPU is configured with only these limited processing functions, and therefore can operate with less power consumption than the main CPU. These functions of the sub-CPU are executed when the main CPU is in standby mode.

[0028] The main power button 20 is an input unit that receives operational input from the user, is provided on the front of the housing of the information processing device 10, and is operated to turn on or off the power supply to the main system 60 of the information processing device 10. The power ON LED 21 lights up when the main power button 20 is turned on, and the standby LED 22 lights up when the main power button 20 is turned off. The system controller 24 detects that the main power button 20 has been pressed by the user.

[0029] The clock 26 is a real-time clock that generates current date and time information and supplies it to the system controller 24 , the subsystem 50 , and the main system 60 .

[0030] The device controller 30 is configured as an LSI (Large-Scale Integrated Circuit) that, like a southbridge, transfers information between devices. As shown in the figure, devices such as the system controller 24, media drive 32, USB module 34, flash memory 36, wireless communication module 38, wired communication module 40, subsystem 50, and main system 60 are connected to the device controller 30. The device controller 30 absorbs differences in the electrical characteristics and data transfer speeds of each device and controls the timing of data transfer.

[0031] The media drive 32 is a drive device that is driven by loading a ROM medium 44 on which application software such as games and license information are recorded, and that reads programs, data, and the like from the ROM medium 44. The ROM medium 44 is a read-only recording medium such as an optical disk, a magneto-optical disk, or a Blu-ray disk.

[0032] The USB module 34 is a module that connects to external devices via a USB cable. The USB module 34 may also be connected to the auxiliary storage device 2 and the camera 7 via USB cables. The flash memory 36 is an auxiliary storage device that constitutes internal storage. The wireless communication module 38 wirelessly communicates with the input device 6 using a communication protocol such as Bluetooth (registered trademark) or IEEE802.11. The wired communication module 40 communicates with external devices via a wired connection and connects to the network 3 via the AP 8.

[0033] The information processing device 10 according to the embodiment executes a game program based on user operations, outputs game images and game sounds to the output device 4, and has a function of automatically generating user-original content including game images and / or game sounds. The information processing device 10 also has a function of automatically performing a process for sharing content with another user. The following describes the function of the information processing device 10 for automatically generating content and the function of automatically performing a process for sharing content.

[0034] <Function for Automatically Generating Content> FIG. 3 shows an example of functional blocks of the information processing device 10. The information processing device 10 according to the embodiment includes a processing unit 100 and a communication unit 102. The processing unit 100 has a function for automatically generating user-original content including game images and / or game sounds. The processing unit 100 includes a game execution unit 110, a game image and sound generation unit 112, an output processing unit 114, a game image acquisition unit 120, a game image identification unit 122, a content storage unit 124, a content presentation unit 126, a learning unit 130, and a user voice acquisition unit 140. The storage device 150 includes a user-generated content storage unit 152 that stores content manually generated by the user in the past (user-generated content), and a content storage unit 154 that stores content automatically generated by the processing unit 100. The storage device 150 may be an auxiliary storage device 2.

[0035] The functionality of the components in the information processing device 10 may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuits, and / or combinations thereof, configured or programmed to perform the functions described herein. A processor is considered to be circuitry or processing circuitry including transistors and other circuits. A processor may also be a programmed processor that executes a program stored in a memory.

[0036] In this specification, a circuit, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions.

[0037] If the hardware is a processor that is considered to be a type of circuitry, the circuitry, means, or unit may be a combination of hardware and software used to configure the hardware and / or processor.

[0038] The communication unit 102 has the functions of both the wireless communication module 38 and the wired communication module 40. While the user is playing a game, the communication unit 102 receives information (game operation information) about the user's operation of the operation members of the input device 6, and provides the information to the game execution unit 110.

[0039] The game execution unit 110 executes game software based on game operation information. The game software includes at least a game program, image data, and sound data. While the user is playing the game, the game execution unit 110 performs calculations to move a player character in a virtual space based on the user's operation information. The game image and sound generation unit 112 includes a GPU (Graphics Processing Unit) and receives the results of calculations in the virtual space to generate game images from a viewpoint position (virtual camera) in the virtual space. The game image and sound generation unit 112 also generates game sounds from the viewpoint position in the virtual space. The output processing unit 114 outputs the game images and game sounds generated by the game image and sound generation unit 112 from the output device 4.

[0040] 4 shows an example of a game image displayed on the output device 4. The user operates the input device 6 while viewing the game images and game sounds output from the output device 4 to progress through the game.

[0041] The user-generated content storage unit 152 stores content (user-generated content) that has been manually generated by a user in the past. In an embodiment, the user-generated content is used as training data for training a content generation model, which is a machine learning model (or an artificial intelligence model). The user-generated content may be used as training data in supervised learning. The learning unit 130 may train the content generation model using user-generated content generated by one user as training data.

[0042] The content generation model is trained to input an image of a game currently being played by a user and output a score value representing the degree to which the game image reflects the characteristics of the user-generated content (similarity). The similarity indicates the degree to which the game image matches the user-generated content, and the score value may be expressed on a scale of 100 points. The closer the score value is to 100 points, the more likely the game image matches the user-generated content, and the closer the score value is to 0 points, the less likely the game image matches the user-generated content.

[0043] When the user-generated content includes still images (screenshots) and moving images, the learning unit 130 may separately train a content generation model for still images and a content generation model for moving images. That is, the content generation model for still images learns the still images, which are user-generated content, to extract features of the user-generated content, and the content generation model for moving images learns the moving images, which are user-generated content, to extract features of the user-generated content. In a modified example, the learning unit 130 may input the still images and moving images, which are user-generated content, into the content generation model without distinguishing between them, and train the content generation model.

[0044] The learning unit 130 may train a content generation model for each game title or game genre. Since the type of game image differs for each game title or game genre, it is preferable that the learning unit 130 trains a content generation model for each game title or game genre.

[0045] During game play, the game image acquisition unit 120 acquires game images and game sounds generated by the game image and sound generation unit 112 and supplies them to the game image identification unit 122. The game image identification unit 122 performs image analysis of the game images acquired by the game image acquisition unit 120 to identify game images related to past user-generated content. In this embodiment, the game image identification unit 122 performs AI image analysis of the game images using a trained content generation model. When an image of a game being played by a user is input, the trained content generation model outputs the degree to which the game image reflects the characteristics of the user-generated content (similarity) as a score value. In this case, the content generation model may output the score value together with information identifying the game image. The information identifying the game image may be the game image itself, or may be metadata such as information indicating the date and time the game image was generated.

[0046] The game image specification unit 122 determines whether a game image is appropriate as content based on the score value output by the content generation model. For example, the game image specification unit 122 may determine that a game image for which a score value equal to or greater than a predetermined score threshold (e.g., 80 points) is appropriate as user-generated content, and may determine that a game image for which a score value less than the score threshold is inappropriate as user-generated content. The content generation model may be configured to output information indicating whether the game image is appropriate or inappropriate as content, and information identifying the game image.

[0047] The following describes a case where the game image identification unit 122 identifies an appropriate still image (screenshot) as content. FIG. 5 shows an example of a flowchart for automatically generating content. The automatic content generation process is performed while the user is playing the game (Y in S10). FIG. 6 shows an example of a game image displayed on the output device 4. The game image acquisition unit 120 acquires the game image generated by the game image and sound generation unit 112 and supplies it to the game image identification unit 122 (S12). The game image acquisition unit 120 may include a buffer memory for temporarily storing the acquired game image. The game image identification unit 122 inputs the supplied game image into a content generation model for still images, performs AI image analysis, and acquires a score value for the game image.

[0048] If the score value of the game image is less than a predetermined score threshold (N in S14), the game image specification unit 122 determines that the game image is not related to past user-generated content, that is, that the game image does not share characteristics with the user-generated content and is therefore inappropriate as content. On the other hand, if the score value of the game image is equal to or greater than a predetermined score threshold (Y in S14), the game image specification unit 122 determines that the game image is related to past user-generated content, that is, that the game image shares characteristics with the user-generated content and is therefore appropriate as content.

[0049] When the game image identification unit 122 determines that an image of the game currently being played by the user is related to past user-generated content, the content saving unit 124 saves the identified game image in the content storage unit 154 as new content (S16). For example, if the score value of the game image shown in FIG. 6 is equal to or greater than a predetermined score threshold, the content saving unit 124 automatically saves the game image shown in FIG. 6 as new content in the content storage unit 154. In this way, the game image identification unit 122 identifies game images that share common features with past screenshots, so that screenshots that reflect the user's preferences are automatically saved in the content storage unit 154. Since screenshots are generated without the user having to issue a command to take a screenshot, the user can concentrate on playing the game. The automatic content generation process may be performed until the user finishes playing the game (N in S10).

[0050] The game image identification unit 122 may identify multiple game images that are similar to one another as game images related to past screenshots. For example, the game images displayed before and after the game image shown in FIG. 6 are similar to one another and often share common features with the past screenshots. When multiple game images that are similar to one another are identified, it is preferable for the content storage unit 124 to store only some of the multiple game images in the content storage unit 154. For example, the content storage unit 124 may store only one of the multiple game images in the content storage unit 154. This makes it possible to avoid storing a large number of similar game images as content, and allows for efficient use of the capacity of the content storage unit 154.

[0051] For this purpose, the content storage unit 124 may have a function for evaluating the degree of similarity of game images successively identified by the game image identification unit 122. It is known that the degree of similarity between two images is evaluated by calculating the distance between the two images, and the content storage unit 124 may identify similar game images using a known evaluation method. If the content storage unit 124 stores a game image as new content in the content storage unit 154 each time the game image identification unit 122 identifies a game image, the content storage unit 124 may store the game image in the content storage unit 154, and then identify multiple similar game images, keep only some of the game images, and delete the remaining game images from the content storage unit 154.

[0052] In the above example, the game image specification unit 122 inputs all generated game images (frame images) into a content generation model for still images and performs AI image analysis. In a modified example, the game image specification unit 122 may input game images into a content generation model for still images at a predetermined cycle and perform AI image analysis. In other words, the game image specification unit 122 may input game images into a content generation model for still images at a frequency of, for example, one image every few seconds and perform AI image analysis.

[0053] During AI image analysis, the game image specification unit 122 may reduce the amount of calculation by lowering the resolution of the game image. The game image specification unit 122 determines whether or not it is necessary to reduce the amount of calculation based on available computational resources, and if it is necessary, may take measures such as lowering the resolution of the game image or canceling the automatic content generation process. If the automatic content generation process is canceled, it is preferable that the game image specification unit 122 notify the user of the reason.

[0054] Next, a case where the game image identification unit 122 identifies an appropriate video as content will be described. The automatic content generation process is performed while the user is playing a game (Y in S10). The game image acquisition unit 120 acquires the game images generated by the game image and sound generation unit 112 and stores them in a buffer memory (S12). The buffer memory may be configured as a ring buffer that stores game video for a predetermined maximum time period (e.g., one hour). The game image identification unit 122 reads out only the predetermined time's worth of game video temporarily stored in the buffer memory, inputs the game video for content generation model for video, performs AI image analysis, and acquires a score value for the game video.

[0055] If the score value of the game video is less than a predetermined score threshold (N in S14), the game image specification unit 122 determines that the game video is not related to past user-generated content, i.e., that the game video does not share common features with the user-generated content and is therefore inappropriate as content. On the other hand, if the score value of the game video is equal to or greater than a predetermined score threshold (Y in S14), the game image specification unit 122 determines that the game video is related to past user-generated content, i.e., that the game video shares common features with the user-generated content and is therefore appropriate as content.

[0056] When the game image identification unit 122 determines that the game video temporarily stored in the buffer memory is related to past user-generated content, the content saving unit 124 saves the identified game video as new content in the content storage unit 154 (S16). By identifying game video that shares common features with past video content in this way, the content storage unit 154 automatically saves game video that reflects the user's preferences. Because the game video is generated without the user issuing a command to capture the game video, the user can concentrate on playing the game. The automatic content generation process may be performed until the user finishes playing the game (N in S10).

[0057] In the above example, the game image specification unit 122 reads out a predetermined amount of game video temporarily stored in the buffer memory and inputs the video content generation model. In another example, the game image specification unit 122 may refer to metadata included in the game video temporarily stored in the buffer memory to determine the start and end points of the game video to be extracted, and then read out the game video between the start and end points. For example, if metadata indicating the start and end of an event (e.g., a boss battle) set in the game video is set, the game image specification unit 122 can determine the start and end points of the game video to be extracted by referring to the metadata. Furthermore, if the metadata indicates that the player character is using a rare weapon, the game image specification unit 122 may read out the game video when the rare weapon is being used.

[0058] The game image identification unit 122 may detect display content and / or display mode included in the game video temporarily stored in the buffer memory, determine the start and end points of the game video to be extracted, and read the game video between the start and end points. For example, if the game video displays "Mission Completed," the game image identification unit 122 may set the time when the content is displayed as the end point, and set a time a predetermined time prior to the end point as the start point, and read the game video between the start and end points. Furthermore, if the game video includes letterboxes (black bars displayed at the top and bottom of the screen), the game image identification unit 122 may read the game video including the letterboxes. The game image identification unit 122 may determine the start and end points of the game video to be extracted using known pattern recognition technology or scene detection technology.

[0059] In addition, the game image identification unit 122 may read a predetermined amount of game video from the buffer memory, input various sections of the predetermined amount of game video into a content generation model for video, perform AI image analysis, and obtain a score value for the game video for each section.

[0060] 7A schematically shows a game video for a predetermined time T. The game image specification unit 122 extracts the game video at various start and end points, inputs the extracted game video into the content generation model, and obtains a score value. The game image specification unit 122 compares the score values ​​of the game video extracted from various sections, and specifies the game video with the highest score value.

[0061] 7B shows an example of a section of the game video showing the highest score value. In this example, the time t １ and time t ２ From this result, the game image specification unit 122 determines that the game moving image between the time t １ is the starting point, and time t ２ is determined as the end point, and the content storage unit 124 determines the time t １ and time t ２ The game moving image between the two may be stored in the content storage unit 154 as new content.

[0062] In an embodiment, the learning unit 130 may periodically train the content generation model using user-generated content that has not yet been learned. For example, the learning unit 130 may train the content generation model once on the last day of a month using the user-generated content generated in that month. The learning unit 130 may train the content generation model every time new user-generated content is generated, or every time a predetermined number of new user-generated content is generated. The learning unit 130 may train the content generation model using content automatically generated by the game image identification unit 122, i.e., content stored in the content storage unit 154. In this case, the learning unit 130 may train the content generation model using automatically generated content that the user has highly rated and / or shared with friends. Information indicating whether the content is user-generated content or automatically generated content is attached to the content as metadata, and the learning unit 130 may train the content generation model so that the learning results of user-generated content are weighted higher than the learning results of automatically generated content.

[0063] In the embodiment, the game image specification unit 122 automatically generates game image content that reflects the user's preferences using a trained content generation model that has learned from user-generated content. In a modified example, the game image specification unit 122 may automatically generate game image content using a trained content generation model that has learned from content previously generated by a user different from the current user. The learning unit 130 collects content previously generated by other users (players) via the network 3 and trains the content generation model.

[0064] The learning unit 130 may train the content generation model using content manually captured by a friend. The learning unit 130 may train the content generation model using content manually captured by players who frequently play a game together. The learning unit 130 may train the content generation model using content manually captured by players who frequently share content. The learning unit 130 may train the content generation model using content manually captured by players who play the same game title. The learning unit 130 may train the content generation model using content manually captured by players who play the same game genre.

[0065] The learning unit 130 allows the content generation model to learn using content generated by another user, which enables the content generation model to generate content that the other user prefers. Therefore, the game image identification unit 122 identifies game images related to content previously generated by a user other than the user from among images of the game being played by the user. The user may be able to set what type of content the learning unit 130 will learn.

[0066] When a game assigns metadata to game video images, the metadata may include, for example, data indicating enemy characters, data indicating weapons used by the player character, a current position on a map, etc. The learning unit 130 may generate a machine learning model for each matching combination of metadata (e.g., enemy character, weapon used, current position), and when the game image identification unit 122 analyzes game images being played by a user, it may perform AI image analysis using a machine learning model with a matching or similar combination of metadata.

[0067] In the embodiment, game image identification unit 122 performs image analysis on images of a game currently being played by the user to identify game images related to past user-generated content, but it may also perform image analysis on images of games previously played by the user to identify game images related to past user-generated content. For example, if storage device 150 stores images of games previously played by the user (note that these images are not user-generated content), game image acquisition unit 120 may acquire the images of the games from storage device 150 and provide them to game image identification unit 122, and game image identification unit 122 may perform image analysis on the game images to identify game images related to past user-generated content.

[0068] The game image specification unit 122 may set usage rights for the automatic content generation function according to the user's account information in the information processing system 1. For example, if the user is a paid member of a network service, the game image specification unit 122 permits the user to use the automatic content generation function, whereas if the user is not a paid member, the game image specification unit 122 does not permit the user to use the automatic content generation function. Note that if grades are set for paid members, use of the automatic content generation function may be partially restricted if the grade is low.

[0069] In the embodiment, the game image identification unit 122 identifies new content by image analysis of game images, but the game image identification unit 122 may also identify game images related to past content by analyzing user voice or game sounds. In the information processing device 10, the user voice acquisition unit 140 acquires voice uttered by the user and supplies it to the game image identification unit 122. When the game image identification unit 122 analyzes the user voice or game sounds and detects excitement, it may cut out a game video using a timing before the section in which the excitement was detected as the start point and a timing after the section as the end point. For example, when the user is yelling loudly, it is highly likely that the user is excited, so the game image identification unit 122 may also take the user voice into consideration when cutting out a game video.

[0070] The content presenting unit 126 may present the game image identified by the game image specifying unit 122 to the user. For example, the content presenting unit 126 may provide the output processing unit 114 with a question message asking, "Do you want to save this game image?" along with the game image, and the output processing unit 114 may display the game image identified by the game image specifying unit 122 and the question message superimposed on the currently displayed game image. The user may select "Yes" or "No" on the screen in response to the question message. If the user selects "Yes," the game image may be saved in the content storage unit 154 as new content. If the user selects "No," the game image may be discarded without being saved in the content storage unit 154. In this case, the user may select whether to save the game image in the content storage unit 154 as new content or to discard the game image by uttering "Yes" or "No." Note that a game image for which "Yes" was selected is one that the user has determined to be appropriate as content, and therefore the learning unit 130 may cause the content generation model to learn the game image as an appropriate game image for user-generated content. On the other hand, for game images for which "No" was selected, the user judged the game image to be inappropriate as content, and therefore the learning unit 130 may have the content generation model learn the game image as a game image that is inappropriate as user-generated content.

[0071] In the information processing system 1, the initial learning model may be provided, for example, from the management server 5, and the learning unit 130 may customize the content generation model by learning user-generated content.

[0072] <Function for Automatically Sharing Content> FIG. 8 illustrates another example of functional blocks of the information processing device 10. The information processing device 10 according to the embodiment includes a processing unit 100 and a communication unit 102. The processing unit 100 has a function for automatically sharing content including game images and / or game sounds. The processing unit 100 includes a game image acquisition unit 210, a content identification unit 212, a sharing processing unit 214, a learning unit 220, and an output processing unit 114. The processing unit 100 according to the embodiment further includes a functional block (see FIG. 3 ) for automatically generating content. The storage device 150 includes a user-generated content storage unit 152 that stores content manually generated by the user in the past (user-generated content), and a content storage unit 154 that stores content automatically generated by the processing unit 100 shown in FIG. 3 . The storage device 150 may be the auxiliary storage device 2. The communication unit 102 has the functions of both the wireless communication module 38 and the wired communication module 40.

[0073] The functionality of the components in the information processing device 10 may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuits, and / or combinations thereof, configured or programmed to perform the functions described herein. A processor is considered to be circuitry or processing circuitry including transistors and other circuits. A processor may also be a programmed processor that executes a program stored in a memory.

[0074] In this specification, a circuit, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions.

[0075] If the hardware is a processor that is considered to be a type of circuitry, the circuitry, means, or unit may be a combination of hardware and software used to configure the hardware and / or processor.

[0076] A user selects content from the storage device 150 that they want to share with other users, and shares the content with other users by uploading the content to the content sharing server 9 or providing it to a specific friend. Metadata indicating that the content has been shared is added to the shared content (hereinafter also referred to as "shared content"). Note that the storage device 150 may separately store information (identification information) that identifies the shared content in a storage unit (not shown) of the storage device 150, and manage the identification information of the shared content and the date and time it was shared.

[0077] In the embodiment, the shared content is used as training data for training a shared content identification model, which is a machine learning model (or an artificial intelligence model). The shared content may be used as training data in supervised learning. The training unit 220 may train the shared content identification model using the shared content shared by one user as training data.

[0078] The shared content identification model is trained to input content, which is an image of a game that a user has played in the past, and output a score value representing the degree to which the content reflects the characteristics of the shared content (similarity). The similarity indicates the degree to which the content matches the shared content, and the score value may be expressed on a scale of 100 points. The closer the score value is to 100 points, the higher the probability that the content matches the shared content, and the closer the score value is to 0 points, the lower the probability that the content matches the shared content.

[0079] When the shared content includes still images (screenshots) and moving images, the learning unit 220 may separately train a shared content identification model for still images and a shared content identification model for moving images. That is, the shared content identification model for still images learns the still images that are the shared content and extracts features of the shared content, and the shared content identification model for moving images learns the moving images that are the shared content and extracts features of the shared content. In a modified example, the learning unit 220 may input the still images and moving images that are the shared content into the shared content identification model without distinguishing between them, and train the shared content identification model.

[0080] The learning unit 220 may train a shared content identification model for each game title or game genre. Since the type of game image varies depending on the game title or game genre, it is preferable that the learning unit 220 generates a shared content identification model for each game title or game genre.

[0081] The game image acquisition unit 210 acquires content that has never been shared from among the content (game images) stored in the storage device 150, and supplies the content to the content identification unit 212. If metadata indicating that the shared content has been shared is attached to the shared content, the game image acquisition unit 210 acquires content to which the metadata is not attached. The game image acquisition unit 210 may acquire screenshots or game moving images that have never been shared.

[0082] The content identification unit 212 performs image analysis of the content acquired by the game image acquisition unit 210 to determine whether the content is related to content previously shared with other users. In this embodiment, the content identification unit 212 performs AI image analysis of the content using a trained shared content identification model. When the trained shared content identification model receives input content, it outputs a score value indicating the degree to which the content reflects the characteristics of the shared content (similarity). In this case, the shared content identification model may output the score value together with information identifying the content. The information identifying the content may be the content itself, or may be information indicating the file name and / or address of the content in the storage device 150.

[0083] The content identification unit 212 determines whether or not content is appropriate as shared content based on the score value output by the shared content identification model. For example, the content identification unit 212 may determine that content for which a score value equal to or greater than a predetermined score threshold (e.g., 80 points) is appropriate as shared content, and may determine that content for which a score value less than the score threshold is appropriate as shared content. Note that the shared content identification model may be configured to output information indicating whether the content is appropriate or inappropriate as shared content and information identifying the content.

[0084] FIG. 9 shows an example of a flowchart for content sharing processing. When unshared content exists in the storage device 150 (Y in S20), the content sharing processing shown in FIG. 9 is performed. Note that the unshared content may be shared on the condition that it has never been evaluated by this flow. The processing unit 100 may perform the content sharing processing each time new content is stored in the storage device 150, or may perform the content sharing processing each time a predetermined number of new content items are stored in the storage device 150.

[0085] The game image acquisition unit 210 acquires unshared content from the storage device 150 and supplies it to the content identification unit 212 (S22). This content is an image of a game that the user has played in the past. The content identification unit 212 inputs the supplied content into a shared content identification model, performs AI image analysis, and acquires a score value for the content. When a shared content identification model for still images and a shared content identification model for moving images exist, if the supplied content is a still image, the content identification unit 212 inputs the content into the shared content identification model for still images to acquire a score value for the content, and if the supplied content is a moving image, the content identification unit 212 inputs the content into the shared content identification model for moving images to acquire a score value for the content.

[0086] If the score value of the content is less than a predetermined score threshold (N in S24), the content identification unit 212 determines that the content is not related to past shared content, i.e., that the content does not have characteristics in common with past shared content and is therefore unsuitable as shared content. On the other hand, if the score value of the content is equal to or greater than a predetermined score threshold (Y in S24), the content identification unit 212 determines that the content is related to past shared content, i.e., that the content has characteristics in common with past shared content and is therefore suitable as shared content.

[0087] When the content identification unit 212 determines that the content read from the storage device 150 is related to the previously shared content, the sharing processing unit 214 starts processing to share the content with other users. Specifically, the sharing processing unit 214 starts processing to directly provide the content to other users or upload the content to the content sharing server 9 so that other users can view the content. The sharing processing unit 214 may present the content determined to be related to the previously shared content to the user from the output processing unit 114 (S26).

[0088] 10 shows an example of a screen presenting candidate content for sharing. The sharing processing unit 214 may present candidate content for sharing and options for sharing destinations to the user and ask the user whether or not to share the content. It is preferable that the sharing processing unit 214 presents candidate content for sharing to the user when the user is not playing a game. Therefore, while the user is playing a game, the sharing processing unit 214 does not present candidate content for sharing to the user so as not to interrupt the gameplay, and presents candidate content for sharing to the user after the gameplay has ended. The candidate content for sharing may be screenshots or game video images.

[0089] If the user wants to share the content with other users, the user selects "Yes," and if not, the user selects "No." In this example, the user operates the input device 6 to select a "Yes" or "No" display area, but the user may also be able to choose whether or not to share the content with other users by vocalizing "Yes" or "No." If sharing, the user preferably can specify a sharing destination. In this example, "content sharing server" is selected. If sending content directly to friends, checking "Select friends" displays a friend list, allowing the user to select one or more friends with whom to share the content. Multiple options for "content sharing server" may be provided, allowing the user to select one or more content sharing servers. The user may also be able to select one or more friends and one or more content sharing servers as sharing destinations.

[0090] If the user selects "Yes," the sharing processing unit 214 determines that the user has agreed to share the content (Y in S28) and transmits the content to the specified sharing destination (S30). This allows the user to share content with other users without manually selecting the content. Furthermore, if the content in question is content that the user has missed out on sharing, the sharing processing of the embodiment can provide the user with an opportunity to share the content.

[0091] On the other hand, if the user selects "No," the sharing processing unit 214 determines that the user has not consented to the content sharing process (N in S28), and the content is not shared with other users. The content sharing process ends when it has been performed on all unshared content (N in S20).

[0092] In the flowchart shown in FIG. 9 , the sharing processing unit 214 presents candidate content to the user to obtain the user's consent. In a modified example, the sharing processing unit 214 may share content without presenting candidate content to the user. In this case, the sharing processing unit 214 autonomously determines a sharing destination and transmits the content to the sharing destination. The sharing processing unit 214 may determine the sharing destination that has been selected most frequently in the past as the sharing destination of the content. For example, if past statistics show that screenshots tend to be shared with specific users and videos tend to be uploaded to a content sharing server, the sharing processing unit 214 may determine the sharing destination of the content based on the past statistical data.

[0093] In the example screen shown in FIG. 10 , the sharing processing unit 214 presents one content item to the user as a candidate for sharing. However, multiple content items may be presented to the user as candidate for sharing. FIG. 11 shows another example of a screen presenting candidate content items. The sharing processing unit 214 may present multiple candidate content items to the user and ask the user whether or not to share the content items. Although five image frames are shown in FIG. 11 , content items may actually be displayed in each image frame. The candidate content items may be screenshots or game animations. The user selects the content items to share by checking the checkboxes corresponding to the content items they wish to share. After selecting the content items, the user can share the content items with other users by selecting "Yes." In this example, the user operates the input device 6 to select a "Yes" or "No" display area. However, the user may also be able to choose whether or not to share the content items with other users by vocalizing "Yes" or "No."

[0094] In an embodiment, the learning unit 220 may periodically train the shared content identification model using shared content that has not yet been learned from among the content (game images) stored in the storage device 150. For example, the learning unit 220 may train the shared content identification model once on the last day of a month using content shared in that month. Note that the learning unit 220 may train the shared content identification model every time new content is shared, or every time a predetermined number of new content items are shared.

[0095] In the embodiment, the content identification unit 212 automatically identifies content to be shared using a trained shared content identification model that has learned content shared by a user. In a modified example, the content identification unit 212 may automatically identify content to be shared using a trained shared content identification model that has learned content shared by a user different from the user. The learning unit 220 collects content shared by other users (players) via the network 3 and trains the shared content identification model.

[0096] The learning unit 220 may train the shared content identification model using content shared by friends. The learning unit 220 may train the shared content identification model using content shared by players who frequently play a game together. The learning unit 220 may train the shared content identification model using content shared by players who frequently share content. The learning unit 220 may train the shared content identification model using content shared by players who play the same game title. The learning unit 220 may train the shared content identification model using content shared by players who play the same game genre. The learning unit 220 may also train the shared content identification model using content shared by the same sharing destination. In particular, when there are multiple content sharing servers as sharing destinations and each content sharing server has its own unique type of distributed content, the learning unit 220 may train a shared content identification model for each content sharing server.

[0097] The learning unit 220 trains the shared content identification model using content shared by another user, so that the shared content identification model can identify content that the other user prefers to share. Therefore, the content identification unit 212 can identify, from among the user's content, content related to content shared by users other than the user. The user may be able to set what type of content the learning unit 220 is to learn.

[0098] When a game assigns metadata to game video images, the metadata may include, for example, data indicating an enemy character, data indicating a weapon used by a player character, a current position on a map, etc. The learning unit 220 may generate a machine learning model for each matching combination of metadata (e.g., enemy character, weapon used, current position), and when the content identification unit 212 analyzes content, it may perform AI image analysis using a machine learning model with a matching or similar combination of metadata.

[0099] In the embodiment, the content identification unit 212 performs image analysis on an image of a game that the user has played in the past to determine whether the image is a game image related to content that the user has shared with other users in the past, but the content identification unit 212 may also perform image analysis on an image of a game that the user is currently playing to determine whether the image is a game image related to content that the user has shared with other users in the past. For example, the content identification unit 212 may perform image analysis on a game image acquired by the game image acquisition unit 120 shown in FIG. 3 to determine whether the image is a game image related to content that the user has shared with other users in the past.

[0100] In the embodiment, the content identification unit 212 performs image analysis of the content to determine whether the content is related to the shared content, but the content identification unit 212 may also analyze user voice or game sounds to determine whether the content is related to the shared content. In the information processing device 10, the user voice acquisition unit 140 acquires voice uttered by the user and supplies the voice to the content identification unit 212. When the content identification unit 212 analyzes the user voice or game sounds and detects excitement, it may determine that content as a sharing candidate.

[0101] The present disclosure has been described above based on the embodiments. These embodiments are merely examples, and it will be understood by those skilled in the art that various modifications are possible in the combination of the respective components and the respective treatment processes, and that such modifications are also within the scope of the present disclosure.

[0102] In the embodiment, the information processing device 10 includes the processing unit 100 that realizes a content generation function and a content sharing function, but the content generation function and the content sharing function may be realized by cooperation between the information processing device 10 and the management server 5. For example, when the information processing system 1 includes a cloud game server, the content generation function and the content sharing function may be realized by the cloud game server alone, or may be realized by cooperation between the cloud game server and the management server 5, or may be realized by cooperation between the cloud game server and the information processing device 10, or may be realized by cooperation between the cloud game server, the information processing device 10, and the management server 5.

[0103] 9 in the embodiment, unshared content stored in the storage device 150 is the target of the sharing process, but shared content may also be the target of the sharing process. For example, when the number of sharing destinations increases due to the addition of a new friend or the availability of a new content sharing server in the information processing system 1, the shared content may be re-evaluated according to the flow shown in FIG.

[0104] In the embodiment, the learning unit 130 trains the content generation model using user-generated content generated by one user as training data. In a modified example, the management server 5 pre-trains the content generation model using user-generated content generated by a large number of users participating in the information processing system 1 as training data, thereby generating a trained content generation model common to all users (hereinafter also referred to as a “common content generation model”). The management server 5 may provide the common content generation model to the information processing device 10, and the learning unit 130 may fine-tune the common content generation model using user-generated content generated by the user operating the information processing device 10. The learning unit 130 may fine-tune the common content generation model for each game title or game genre, or may fine-tune the common content generation model using user-generated content generated by another user. In a modified example, generating the common content generation model through pre-training allows a highly reliable content generation model to be generated even when there is little user training data.

[0105] In the embodiment, the learning unit 220 trains the shared content identification model using shared content shared by one user as training data. In a modified example, the management server 5 pre-trains the shared content identification model using shared content shared by a large number of users participating in the information processing system 1 as training data, thereby generating a trained shared content identification model common to all users (hereinafter also referred to as a "common content identification model"). The management server 5 provides the common content identification model to the information processing device 10, and the learning unit 220 may fine-tune the common content identification model using shared content shared by the user operating the information processing device 10. The learning unit 220 may fine-tune the common content identification model for each game title or game genre, or may fine-tune using shared content generated by another user. In a modified example, generating the common content identification model through pre-training makes it possible to generate a highly reliable shared content identification model even when there is little user training data.

[0106] The present disclosure may include the following aspects. [Item 1] An information processing system including a circuit configured to: acquire images of a game that a user has played in the past or is currently playing; perform image analysis on the acquired game images to determine whether the game images are related to content previously shared with other users; and share the game images determined to be related to the previously shared content with another user. [Item 2] The information processing system according to item 1, wherein the circuit determines whether the acquired game images have common features with content previously shared with other users. [Item 3] The information processing system according to item 1, wherein the circuit determines a sharing destination for the game images. [Item 4] The information processing system according to item 1, wherein the circuit presents the game images determined to be related to the previously shared content to the user. [Item 5] The information processing system according to item 4, wherein the circuit presents options for sharing destinations to the user along with the game images. [Item 6] The information processing system according to item 4, wherein the circuit presents the game images to the user when the user is not playing the game. [Item 7] The information processing system according to Item 1, wherein the circuit acquires screenshots or game video images as game images. [Item 8] A method for sharing game images, comprising: acquiring images of a game that a user has played in the past or images of a game that the user is currently playing; analyzing the acquired game images to determine whether the game images are related to content that has been shared with another user in the past; and allowing the game images determined to be related to the previously shared content to be shared with another user.[Item 9] A recording medium having recorded thereon a program executed on a computer of an information processing device, the program causing the computer to realize the following functions: acquiring images of a game that a user has played in the past or images of a game that the user is currently playing; analyzing the acquired game images to determine whether or not the game images are related to content that the user has shared with other users in the past; and allowing game images that have been determined to be related to previously shared content to be shared with other users.

[0107] The present disclosure can be used in the technical field of saving images of a game that a user is currently playing or images of a game that a user has played in the past as content and / or the technical field of sharing content with other users.

[0108] 1...information processing system, 10...information processing device, 100...processing unit, 102...communication unit, 110...game execution unit, 112...game image and sound generation unit, 114...output processing unit, 120...game image acquisition unit, 122...game image identification unit, 124...content storage unit, 126...content presentation unit, 130...learning unit, 140...user voice acquisition unit, 150...storage device, 152...user-generated content storage unit, 154...content storage unit, 210...game image acquisition unit, 212...content identification unit, 214...sharing processing unit, 220...learning unit.

Claims

1. An information processing system, comprising: a game image acquisition unit that acquires an image of a game that a user has played in the past or an image of a game that the user is currently playing; a content identification unit that performs image analysis on the acquired game image to determine whether it is a game image related to content shared with other users in the past; and a sharing processing unit that causes a game image determined to be related to past shared content to be shared with another user. The information processing system is characterized by comprising these components.

2. The information processing system according to claim 1, wherein the content identification unit determines whether the acquired game image has common features with content shared with other users in the past.

3. The information processing system according to claim 1, wherein the sharing processing unit determines a sharing destination for the game image.

4. The information processing system according to claim 1, wherein the sharing processing unit presents a game image determined to be related to past shared content to the user.

5. The information processing system according to claim 4, wherein the sharing processing unit presents options for sharing destinations to the user together with the game image.

6. The information processing system according to claim 4, wherein the sharing processing unit presents the game image to the user when the user is not playing a game.

7. The information processing system according to claim 1, wherein the game image acquisition unit acquires a screenshot or a game video as the game image.

8. A method for sharing game images in an information processing system, comprising: acquiring an image of a game that a user has played in the past or an image of a game that the user is currently playing; performing image analysis on the acquired game image to determine whether it is a game image related to content shared with other users in the past; and causing a game image determined to be related to past shared content to be shared with another user. The method for sharing game images is characterized by comprising these steps.

9. A program for causing a computer to implement a function of acquiring an image of a game that the user has played in the past or an image of a game that the user is playing, a function of analyzing the acquired game image to determine whether it is a game image related to content shared with other users in the past, and a function of sharing the game image determined to be related to past shared content with another user.

Images