Generation system, program, and information processing device

Abstract

To provide a generation system or the like that can generate a user-desired screen capture without failure.SOLUTION: A screen capture instruction is accepted on the basis of a user's input operation. A screen is captured at instruction timing at which the instruction is accepted. It is determined whether or not the screen capture satisfies specified conditions. If it is determined that the specified conditions are not satisfied, content information based on the instruction timing is changed. The screen capture is generated on the basis of the changed content information.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a generation system, a program, and an information processing device. [Background technology]

[0002] It has been expected that images that users like will be generated in virtual spaces. For example, there is a technology in which, on a separate shooting screen from the normal one, characters that the photographer cannot normally control (NPCs (non-player characters) or characters of other users) move to positions suitable for shooting, and while shooting, the normal screen is displayed on the screen of the other users and the game progresses, and after shooting, the latest normal screen is also displayed on the photographer's screen (Patent Document 1).

[0003] Furthermore, there is a technology that determines image generation conditions such as the virtual camera position based on the subject's position when the game situation satisfies certain conditions (for example, clearing a difficult section), and displays an image generated under those conditions along with information related to the subject's game situation (for example, the time required to clear the game and the number of attempts) (Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6144738 [Patent Document 2] Japanese Patent Application Publication No. 2023-107821 Summary of the Invention [Problem to be solved by the invention]

[0005] In some cases, a user may operate a terminal device to capture a screen capture (still image or video) of a screen that provides content. However, the captured screen capture may not be what the user desires.

[0006] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to provide a generation system etc. that can successfully generate a screen capture of a screen that provides the content desired by the user. [Means for solving the problem]

[0007] (1) The present invention provides 1. A generation system for generating a screen capture of a screen providing content, comprising: a receiving unit that receives a screen capture instruction based on an input operation by a user; a capture unit that captures the screen at the instruction timing when the instruction is received; a determination unit that determines whether the screen capture captured by the capture unit satisfies a predetermined condition; a content information control unit that changes content information based on the instruction timing when it is determined that the predetermined condition is not satisfied; and a screen capture generation unit that generates a screen capture based on the changed content information.

[0008] The present invention also relates to an information processing device (terminal device, game device, server device) including each of the above-mentioned units. The present invention also relates to a program that causes a computer to function as each of the above-mentioned units. The present invention also relates to a computer-readable information storage medium, The present invention relates to an information storage medium that stores a program that causes a computer to function.

[0009] Here, the screen capture includes at least one of a still image and a video. According to the present invention, even if the screen capture obtained based on the user's input operation fails, a new screen capture is generated based on the changed content information, so that the screen capture desired by the user can be generated without failure.

[0010] (2) Furthermore, in the generation system, program, and information processing device according to the present invention, The device may further include a condition control unit that sets the predetermined condition based on an input operation by a user.

[0011] According to the present invention, the user can set predetermined conditions, so that a screen capture that satisfies the user can be generated.

[0012] (3) Furthermore, in the generation system, program, and information processing device according to the present invention, The device may further include a condition control unit that sets the predetermined condition based on the user's usage history.

[0013] According to the present invention, it is possible to set predetermined conditions based on the user's play history, and therefore it is possible to generate a screen capture that is suited to the user's usage history.

[0014] (4) Furthermore, in the generation system, program, and information processing device according to the present invention, The content management device may further include a condition control unit that sets the predetermined condition based on content information.

[0015] According to the present invention, predetermined conditions can be set based on content information, so that a screen capture suited to the content information can be generated.

[0016] (5) Furthermore, in the generation system, program, and information processing device according to the present invention, The content information control unit The content information may be changed when a predetermined parameter of the user is consumed.

[0017] According to the present invention, content information is changed on the condition that a predetermined parameter of the user is consumed, so that the value of changing the content information itself can be increased.

[0018] (6) Furthermore, in the generation system, program, and information processing device according to the present invention, The content information may further include a parameter control unit that changes a consumption value consumed from a predetermined parameter of the user in accordance with the degree of change when the content information is changed.

[0019] According to the present invention, when content information is changed on the condition that a user's specified parameters are consumed, the consumption value is changed according to the degree of change in the content information, so that a consumption value commensurate with the change can be set.

[0020] (7) Furthermore, in the generation system, program, and information processing device according to the present invention, The content information may further include a presentation unit that presents the changed content information to a user.

[0021] According to the present invention, by presenting the changed content information to the user, it is possible to increase transparency and user satisfaction.

[0022] (8) Furthermore, in the generation system, program, and information processing device according to the present invention, a presentation unit that presents the screen capture generated by the screen capture generation unit to the user; The screen capture generating device may further include an evaluation unit that allows the user to evaluate the screen capture generated by the screen capture generating unit.

[0023] According to the present invention, it is possible to obtain user ratings of generated screen captures.

[0024] (9) Furthermore, in the generation system, program, and information processing device according to the present invention, The screen capture generating unit may further include a condition control unit that further changes the predetermined condition based on the user's evaluation of the screen capture generated by the screen capture generating unit.

[0025] According to the present invention, the predetermined conditions are further changed based on the user's evaluation, so that the user's evaluation can be effectively utilized.

[0026] (10) Furthermore, in the generation system, program, and information processing device according to the present invention, The content information control unit further modifying the content information based on the user's evaluation of the screen capture generated by the screen capture generator; The screen capture generation unit A screen capture may be generated again based on the changed content information.

[0027] According to the present invention, a new screen capture can be generated based on the changed content information so as to increase user satisfaction.

[0028] (11) Furthermore, in the generation system, program, and information processing device according to the present invention, a learning unit that learns content information at the instruction timing, The screen capture generation unit The screen capture may be generated based on a result of learning by the learning unit.

[0029] According to the present invention, it is possible to generate a screen capture that is optimal for the user in accordance with the timing of the instruction.

[0030] (12) Furthermore, in the generation system, program, and information processing device according to the present invention, If the screen capture is a video, at least one of the duration of the video, the start time of the video, the end time of the video, and the playback data of the audio of the video may be changed.

[0031] According to the present invention, suitable animated screen captures can be generated.

[0032] (13) Furthermore, in the generation system, program, and information processing device according to the present invention, The content information control unit If the screen capture captured by the capture unit does not satisfy the predetermined condition, content information, which is at least one of information about the virtual camera that photographs the object used to generate the screen capture and information about the object, is changed so as to satisfy the predetermined condition; The screen capture generation unit The screen capture may be redrawn.

[0033] According to the present invention, the content information, which is at least one of the information about the virtual camera that captures the object used to generate the screen capture and the information about the object, is changed, so that the screen capture desired by the user can be generated without failure.

[0034] The information about the virtual camera includes, for example, the position, direction, and angle of view of the virtual camera.

[0035] Furthermore, the information about the object may be information about the object itself, such as the position, orientation, parameters, and posture of the object, or may be information related to the object (items equipped by the object) and the like.

[0036] (14) Furthermore, in the generation system, program, and information processing device according to the present invention, The content information control unit When there are multiple items to be changed, the item to be changed with priority may be determined based on a given priority.

[0037] According to the present invention, items that should be changed with priority are determined based on priority conditions, so that a screen capture that meets the user's desires can be generated without failure.

[0038] (15) Furthermore, in the generation system, program, and information processing device according to the present invention, The determination unit determining whether the screen capture captured by the capture unit is an image drawn in a frame buffer and whether the image satisfies the predetermined condition; The content information control unit If the predetermined condition is not met, the image may be changed so as to meet the predetermined condition.

[0039] According to the present invention, an image that has been determined not to satisfy a predetermined condition is modified so that it satisfies the predetermined condition, so that a screen capture that the user desires can be generated without failure.

[0040] (16) Furthermore, in the generation system, program, and information processing device according to the present invention, The content information control unit A portion of the image that does not satisfy the predetermined condition may be identified, and the image may be changed to an image that satisfies the predetermined condition.

[0041] According to the present invention, a portion that does not satisfy a predetermined condition is identified and changed to an image that satisfies the predetermined condition, so that a screen capture desired by the user can be generated without failure.

[0042] (17) Furthermore, in the generation system, program, and information processing device according to the present invention, The content information control unit The image may be changed to an image that satisfies the predetermined conditions using a given generation AI.

[0043] According to the present invention, a screen capture desired by a user can be generated without failure using a generation AI.

[0044] (18) Furthermore, in the generation system, program, and information processing device according to the present invention, The content information control unit The image may be changed to another image that satisfies the predetermined condition.

[0045] According to the present invention, a user can successfully generate a desired screen capture using another image. [Brief explanation of the drawings]

[0046] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a generation system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram showing functional blocks of a server device according to the embodiment. [Figure 3] FIG. 2 is a diagram showing functional blocks of the terminal device according to the embodiment. [Figure 4] 10A to 10C are explanatory diagrams of instruction timing and screen captures according to the present embodiment. [Figure 5] 10A to 10C are explanatory diagrams of instruction timing and screen captures according to the present embodiment. [Figure 6] 10A to 10C are explanatory diagrams of instruction timing and screen captures according to the present embodiment. [Figure 7] 10A and 10B are explanatory diagrams illustrating an example of drawing an image layer and a text layer overlapping each other in this embodiment. [Figure 8] FIG. 10 is an explanatory diagram of an example of a change in model data according to the embodiment. [Figure 9] 10A and 10B are explanatory diagrams illustrating examples of modifications of images drawn in a frame buffer according to the present embodiment. [Figure 10] 10A and 10B are explanatory diagrams illustrating examples of modifications of images drawn in a frame buffer according to the present embodiment. [Figure 11] FIG. 4 is a diagram showing an example of a presentation screen in the present embodiment. [Figure 12] 1 is a flowchart showing an example of a processing flow according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0047] The present embodiment will be described below. Note that the present embodiment described below does not unduly limit the content of the present invention described in the claims. Furthermore, not all of the configurations described in the present embodiment are necessarily essential constituent elements of the present invention.

[0048] [1] Generation system First, an overview and general configuration of a generation system (for example, a game system) according to this embodiment will be described using Fig. 1. Fig. 1 is a diagram showing an example of a system configuration showing the configuration of the generation system according to this embodiment.

[0049] The server device 10 is an information processing device capable of providing a given service using a terminal device (game device) 20 connected for communication via the Internet (an example of a network).

[0050] The terminal device 20 is an information processing device such as a terminal device, smartphone, mobile phone, PHS, computer, PDA, portable game console, image generating device, etc., and is a device that can be connected to the server device 10 via a network such as the Internet (WAN) or LAN.

[0051] [1.1] Generation system configured in a client-server model As shown in FIG. 1, the generation system of this embodiment is configured such that a server device 10 and terminal devices 20 (for example, terminal devices 20A, 20B, and 20C) are connectable to the Internet (an example of a network).

[0052] By accessing the server device 10 from the terminal device 20, the user can receive content (for example, play a game) based on information transmitted from the server device 10 via the Internet.

[0053] In this embodiment, each content may be provided to the terminal device 20 by one server device 10, or a server system may be constructed by linking a plurality of server devices 10, and each content may be provided to the terminal device 20.

[0054] It should be noted that the various processes relating to the present invention may be performed by the server device 10 alone, or by the terminal device 20 alone.

[0055] [1.2] Terminal equipment The generation system of this embodiment may be realized by a single terminal device having the functions of the server device 10, that is, a device (standalone) that operates independently without relying on other devices such as a server device.

[0056] Furthermore, in this embodiment, the present invention may be realized by the terminal device 20 alone, without being connected to the server device 10. For example, the present invention may be realized by a generation system realized by a plurality of terminal devices 20 through P2P (peer-to-peer) communication.

[0057] A plurality of such terminal devices may be connected by wire or wirelessly, and one terminal device may function as a host (server device 10), and the system may be realized by the plurality of terminal devices.

[0058] The terminal device is not limited to a terminal device, but may be a tablet information terminal device, a personal computer, or a terminal device (casing) installed in an amusement park.

[0059] [1.3] Cloud-based generation system The generation system of this embodiment may also be a cloud-based generation system. For example, in a cloud-based generation system, basic processing (including, for example, content-related processing, drawing processing, etc.) is performed solely by the server device 10 (cloud device), and the terminal device 20 only controls the display of the processing results of the server device 10 (cloud device). The terminal device 20 also transmits information such as input information (operation information) of the controller to the server device 10. In this case, the terminal device 20 may also transmit information other than the input information.

[0060] Furthermore, in this embodiment, except for input, the execution of each function and program of the processing unit 200 of the above-mentioned terminal device 20 may be executed by the server device 10, and the terminal device 20 may provide the above-mentioned content by executing input and image display by streaming.

[0061] The generation system may also store the information stored in the server device 10 and the information stored in the terminal device 20 in a given storage area on the Internet.

[0062] [1.4] Terminal device with multiple controllers In this embodiment, one terminal device 20 may be equipped with multiple controllers (input units 260). When multiple users receive content using their own controllers on one terminal device 20, it is assumed that the multiple users are actually close to each other.

[0063] [1.5] Examples of social content In addition, in this embodiment, the device may function as an SNS (Social Networking Service) server that provides communication-type services. Here, the SNS server may be an information processing device that provides a service that allows communication between multiple users.

[0064] In addition, in this embodiment, when it functions as an SNS server, for example, the SNS It is possible to provide games called social games that are executed using the operating environment of S (API (Application Programming Interface), platform, etc.).

[0065] Unlike existing online games, social games include games that do not require dedicated client software and can be used with just a web browser and an SNS account. In addition, this embodiment has a configuration that makes it possible to provide an online game in which users can connect to other users' terminal devices 20 via a network and share the same game progress simultaneously online.

[0066] [1.6] Browser example In particular, in this embodiment, content provided on the web browser of the terminal device 20, such as browser games (games that can be launched simply by opening the installation site in the web browser) created in various languages ​​such as HTML, FLASH (registered trademark), CGI, PHP, shockwave, Java (registered trademark) applet, and JavaScript (registered trademark), may be provided.

[0067] The terminal device 20 also includes a web browser capable of viewing web pages (HTML data). That is, the terminal device 20 includes a communication control function for communicating with the server device 10, a web browser function for performing display control using data received from the server device 10 (web data, data created in HTML format, etc.), and a web browser function for transmitting user operation data to the server device 10, and is configured to perform various processes for providing a screen to the user and allow the user to execute content. However, the terminal device 20 may also acquire content control information provided from the server device 10, execute predetermined content processing, and provide content based on the content processing.

[0068] Specifically, when the terminal device 20 makes a request to the server device 10 to receive predetermined content, the terminal device 20 is connected to the content site of the server device 10 and starts the content. In particular, the terminal device 20 may be configured to use an API as necessary to cause the server device 10 functioning as an SNS server to perform predetermined processing, or to obtain user information and the like managed by the server device 10 functioning as an SNS server, thereby processing the content.

[0069] [1.7] Other In this embodiment, the server device 10 may be configured with one (device, processor) or multiple (devices, processors). Information (e.g., user information, content information, etc.) stored in a storage area (storage unit 170 described later) of the server device 10 may be stored in a database (broadly speaking, a storage device, a memory) connected via a network (an intranet or the Internet). The communication line between the terminal device 20 and the server device 10 may be wired or wireless.

[0070] [2] Server device Next, the server device 10 of this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing functional blocks of the server device 10 of this embodiment. In this embodiment, some of the components (each unit) of Fig. 2 may be omitted.

[0071] In this embodiment, the system includes an input unit 160 for use in input by the administrator and others, an information storage medium 180 in which predetermined information is stored, a communication unit 196 for communicating with the terminal device 20 and others, a processing unit 100 that mainly executes processing related to the content to be provided, and a storage unit 170 that mainly stores various data used for the content.

[0072] The input unit 160 is used by a system administrator or the like to input settings related to content and other necessary settings and data. For example, the input unit 160 in this embodiment is configured by a mouse, a keyboard, and the like.

[0073] The information storage medium 180 (computer-readable medium) stores programs, data, etc., and its functions are realized by an optical disk (CD, DVD), a magneto-optical disk (MO), a magnetic disk, a hard disk, a magnetic tape, or a memory (ROM), etc.

[0074] The communication unit 196 performs various controls for communicating with the outside (e.g., terminals, other servers, or other network systems), and its functions are configured by hardware such as various processors or communication ASICs, programs, etc.

[0075] The storage unit 170 serves as a work area for the processing unit 100, the communication unit 196, etc., and its functions are realized by RAM (VRAM), etc. The information stored in the storage unit 170 may be managed by a database.

[0076] In addition to the main memory 171, the storage unit 170 of this embodiment also includes an image buffer 172, a content information storage unit 173, and a user information storage unit (player information storage unit) 174.

[0077] In particular, the content information storage unit 173 stores information on the virtual space (for example, game space), information on the virtual camera, information on each character, parameters used in the content, and the like.

[0078] Furthermore, user information (user information) is stored in association with a user ID (user ID) for each user (user) in the user information storage unit 174. The user ID is identification information for identifying a user.

[0079] The user information includes input information (operation information) of the user. The user information may be a concept including not only user information associated with the user but also information on the user's content media (for example, game media such as characters, cards, and items owned by the user).

[0080] The processing unit 100 performs various processes using a main memory unit 171 in the memory unit 170 as a work area. The functions of the processing unit 100 can be realized by hardware such as various processors (CPU, DSP, etc.) and ASICs (gate arrays, etc.), or by programs.

[0081] The processing unit 100 performs various processes of this embodiment based on programs (data) stored in the information storage medium 180. That is, the information storage medium 180 stores programs for causing a computer to function as each unit of this embodiment (programs for causing a computer to execute the processing of each unit).

[0082] For example, the processing unit 100 (processor) controls the entire server device 10 based on a program stored in the information storage medium 180, and performs various processes such as controlling the transfer of data between the various units. Furthermore, the processing unit 100 performs processes to provide various services in response to requests from the terminal device 20.

[0083] The processing unit 100 is linked to the terminal device 20 and receives the information input via the terminal device 20. Based on the user's input, each user executes processing of the content provided in this embodiment.

[0084] In this embodiment, the server device 10 may perform part or all of the processing of the processing unit 100, or the terminal device 20 may perform part of the processing of the processing unit 100.

[0085] The processing unit 100 includes a content control unit 111 , a display control unit 112 , a receiving unit 113 , a presentation unit 120 , a communication control unit 121 , a management unit 122 , an image generation unit 130 , and a sound processing unit 140 .

[0086] The content control unit 111 executes processing related to the content based on input information (operation information) from the user. For example, the content control unit 111 may receive input information (operation information) from the user's terminal device 20 and execute processing related to the content.

[0087] The display control unit 112 controls the display of the images (including still images and moving images) generated by the server device 10 on the display unit 290 of the terminal device 20. In other words, the display destination of the images generated by the server device 10 is the display unit 290 of the terminal device 20.

[0088] The accepting unit 113 accepts information from a user (player). For example, the accepting unit 113 receives and accepts information from the user's terminal device 20 via a network.

[0089] The presentation unit 120 performs processing to present given information to the terminal device 20 of the user.

[0090] The communication control unit 121 establishes a connection (session or connection) with the terminal device 20 and performs processing to communicate (transmit and receive) data via the network.

[0091] For example, the communication control unit 121 transmits information related to content to the user's terminal device 20. The communication control unit 121 also receives, from the user's terminal device 20, user operation information accepted by the terminal device 20. The communication control unit 121 also receives information related to content from the server device 10.

[0092] The management unit 122 manages content information and user information associated with user identification information for each user. The management unit 122 may also manage information such as in-content currency and items.

[0093] The image generation unit 130 performs drawing processing based on the results of various processes performed by the processing unit 100 , thereby generating an image, and outputs it to the display unit 290 of the terminal device 20 .

[0094] The image generation unit 130 is realized by, for example, a processor such as a GPU or a digital signal processor (DSP), a video signal IC, a program such as a video codec, an IC memory for drawing frames such as a frame buffer, and the like.

[0095] The image generation unit 130 generates one image (game screen) of the content screen in one frame time (for example, 1 / 60 seconds) in accordance with the progress of the content. The image generation unit 130 outputs the generated image to the image buffer 172. The image generation unit 130 also transmits the generated image to the terminal device 20.

[0096] The image generation unit 130 generates an image using a virtual camera in a virtual space (object space, game space). It generates an image (a so-called 3D image) that can be seen from a given viewpoint.

[0097] For example, when generating a 3D game image, the image generation unit 130 first performs geometry processing such as coordinate transformation (world coordinate transformation, camera coordinate transformation), clipping, or perspective transformation, and then creates rendering data (position coordinates of vertices of primitive surfaces, texture coordinates, color data, normal vectors, alpha values, etc.) based on the results of these processing. Then, based on this rendering data (primitive surface data), the object (one or more primitive surfaces) after perspective transformation (after geometry processing) is rendered in the image buffer 172 (a buffer that can store image information in pixel units, such as a frame buffer or work buffer; VRAM). This generates an image seen from a virtual camera (a given viewpoint) in object space.

[0098] Furthermore, when multiple users play a multi-user online game in which data is transmitted and received via a network, the image generation unit 130 performs a process for generating an image seen by a virtual camera that follows the movement of an object operated by each user. In other words, an independent rendering process is performed for each user. The image generation unit 130 may also generate a two-dimensional image in the virtual space.

[0099] The sound processing unit 140 performs sound processing based on the results of various processes performed by the processing unit 100, generates (plays) game sounds such as background music (BGM), sound effects, or voices, and transmits them to the terminal device 20.

[0100] [3] Terminal device Next, the terminal device 20 of this embodiment will be described with reference to Fig. 3. Fig. 3 is an example of a functional block diagram showing the configuration of the terminal device 20 of this embodiment. Furthermore, the terminal device 20 of this embodiment may have a configuration in which some of the components (units) of Fig. 3 are omitted.

[0101] The input unit 260 is used by the user to input operation data, and this function can be realized by a touch panel or a touch panel display, etc. That is, the input unit 260 includes a detection unit 262 that can detect two-dimensional pointed position coordinates (x, y) on a screen on which an image is displayed. For example, the input unit 260 includes a detection unit 262 that can detect two-dimensional contact position coordinates (x, y) in a contact detection area (touch panel).

[0102] The touch operation on the display screen (hereinafter referred to as "touch panel" unless otherwise specified) may be performed using a fingertip or an input device such as a touch pen.

[0103] The input unit 260 may also include a button, lever, keyboard, steering wheel, microphone, acceleration sensor, etc., which can input input information (operation signals) other than the indicated position.

[0104] The storage unit 270 serves as a work area for the processing unit 200, the communication unit 296, etc., and its functions can be realized by RAM (VRAM) or the like. The storage unit 270 of this embodiment includes a main storage unit 271 used as a work area, an image buffer 272 in which the final display image and the like are stored, a content information storage unit 273, a user information storage unit 274, a screen capture storage unit 275, an evaluation information storage unit 276, a learning storage unit 278, and a learning model storage unit 279. Note that some of these may be omitted, or the storage unit 170 of the server device 10 may constitute part of them.

[0105] The image buffer 272 may store image data generated by the image generating unit 230 or image data received from the server device 10.

[0106] The content information storage unit 273 stores information on the virtual space (for example, game space), information on the virtual camera, information on each character, parameters used in the content, etc. Note that the terminal device 20 may store information received from the server device 10 in the content data storage unit 273.

[0107] User information is stored in the user information storage unit 276. Note that the terminal device 20 may receive user information corresponding to a user ID from the server device 10 and store the information in the user information storage unit 276.

[0108] The screen capture storage unit 275 stores the screen capture acquired by the capture unit 214, the screen capture after the content information has been changed, and the like.

[0109] The evaluation information storage unit 276 stores information on evaluations of the screen captures generated by the screen capture generation unit 217.

[0110] The learning storage unit 278 stores learning information. The learning storage unit 178 also stores evaluation information. Specifically, the learning storage unit 278 stores video (moving images, image data). This video includes video generated by the video generation unit 230 of this embodiment. It may also include video generated by AI (AI is an abbreviation for Artificial Intelligence). The terminal device 20 may also receive learning information from another information processing device and store it in the learning storage unit 278.

[0111] The learning model storage unit 279 stores a learning model (model). The learning model is a mathematical expression (function) that represents the correlation (relationship) and tendency between input data and output data. The learning model storage unit 279 stores a learning model that has been machine-learned by the learning unit 222.

[0112] The information storage medium 280 (computer-readable medium) stores programs, data, etc., and its functions can be realized by an optical disk (CD, DVD), a magneto-optical disk (MO), a magnetic disk, a hard disk, a magnetic tape, or a memory (ROM), etc.

[0113] Furthermore, a program for causing a computer to function as each unit of this embodiment (a program for causing a computer to execute the processing of each unit) can be stored in the information storage medium 280. Note that the processing unit 200 performs various processes of this embodiment based on the program (data) stored in the information storage medium 280, as will be described later.

[0114] The display unit 290 outputs the image generated by this embodiment, and its function can be realized by a CRT, LCD, touch panel display, HMD (head mounted display), or the like.

[0115] In particular, in this embodiment, the display unit 290 also functions as an input unit 260 that the user operates using a touch panel display. Here, the touch panel may be, for example, a resistive type (four-wire type, five-wire type), a capacitance type, an electromagnetic induction type, an ultrasonic surface acoustic wave type, or an infrared scanning type.

[0116] The sound output unit 292 outputs the sound generated by this embodiment, and its function can be realized by a speaker, headphones, or the like.

[0117] The communication unit 296 performs various controls for communicating with the outside (for example, the server device 10 or other terminal devices 20), and its functions can be realized by hardware such as various processors or communication ASICs, or programs.

[0118] The terminal device 20 may receive, via a network, programs and data for causing a computer to function as each unit of the present embodiment, which are stored in the information storage medium 180 or the storage unit 170 of the server device 10, and store the received programs and data in the information storage medium 280 or the storage unit 270. Such cases in which the terminal device 20 functions by receiving programs and data are also included within the scope of the present invention.

[0119] The processing unit 200 (processor) performs processing such as content processing, image generation processing, or sound generation processing in conjunction with the server device 10 based on input data from the input unit 260, programs, and the like.

[0120] In particular, in this embodiment, content processing includes processing to start content when content start conditions are met, processing to progress content, processing to place objects such as characters and enemy characters, processing to display objects, processing to calculate content results, and processing to end content when content end conditions are met.

[0121] Furthermore, the processing unit 200 performs content processing (for example, processing of a competitive game, etc.) based on user input information input by the input unit 260. Furthermore, the processing unit 200 may be executed in conjunction with the server device 10, or a part or all of the processing unit 200 may be formed in the server device 10.

[0122] The processing unit 200 performs various processes using the storage unit 270 as a work area. The functions of the processing unit 200 can be realized by hardware such as various processors (CPU, DSP, etc.) and ASICs (gate arrays, etc.), or by programs.

[0123] The processing unit 200 includes a content control unit 211, a display control unit 212, a receiving unit 213, a capture unit 214, a determination unit 215, a content information control unit 216, a screen capture generation unit 217, a condition control unit 218, an evaluation unit 219, a communication control unit 221, a learning unit 222, a presentation unit 223, an image generation unit 230, and a sound processing unit 240. Note that some of these units may be omitted.

[0124] The content control unit 211 executes processing related to the content based on input information (operation information) from the user.

[0125] The content control unit 211 progresses the game based on the operation input of the user. For example, the content control unit 211 may progress the game by making the character operated by the user take a predetermined action.

[0126] The display control unit 212 performs processing to display a screen on the display unit 290. For example, the display control unit 211 may display using a web browser. Furthermore, the display control unit 211 causes the display unit 290 to display information required in the content based on information transmitted from the server device 10. For example, the terminal device 20 displays the display information of the screen received from the server device 10.

[0127] Furthermore, the display control unit 212 may perform all or part of the same processing as the display control unit 112 of the server device 10.

[0128] The receiving unit 213 performs a process of receiving input from a user. For example, the receiving unit 212 may transmit input information received by the terminal device 20 to the server device 10, and the server device 10 may receive the input information.

[0129] The receiving unit 213 receives a screen capture instruction based on an input operation by the user.

[0130] The capture unit 214 captures the screen at the timing when the instruction to capture the screen is received.

[0131] For example, the screen may be an image that displays an image seen from a virtual camera in the virtual space. In other words, the capture target is an image (moving image) stored in the frame buffer (drawing buffer 272).

[0132] For example, the capture unit 214 captures a screenshot as a still image at the instruction timing. Also, the capture unit 214 starts capturing a moving image at the instruction timing to start, and ends capturing the moving image at the instruction timing to end. The instruction timing may be rephrased as a shooting timing or a capture timing.

[0133] The determination unit 215 may determine whether or not the screen capture captured by the capture unit 214 satisfies a predetermined condition.

[0134] The determination unit 215 may determine whether or not the screen capture captured by the capture unit 214 is an image drawn in the frame buffer (image buffer 272) and the image satisfies a predetermined condition.

[0135] If it is determined that the predetermined condition is not satisfied, the content information control unit 216 may change the content information based on the instruction timing.

[0136] The content information control unit 216 may change the content information when a predetermined parameter of the user is consumed.

[0137] The content information control unit 216 may further modify the content information based on the user's evaluation of the screen capture generated by the screen capture generation unit 217.

[0138] When the screen capture is a video, the content information control unit 216 may change at least one of the duration of the video, the start point of the video, the end point of the video, and the playback data of the audio of the video.

[0139] When the screen capture captured by the capture unit 214 is an image of an object placed in a virtual space, the content information control unit 216 may acquire content information, which is at least one of the position of the virtual camera capturing the object, the orientation of the virtual camera capturing the object, the posture of the object, and information about the object, at the timing of the capture. For example, the content information at the time of capture may be acquired, or the content information for a period of several frames before the time of capture may be acquired, or the content information for a period of several frames after the time of capture may be acquired.

[0140] If the content information control unit 216 determines that the screen capture does not satisfy a predetermined condition, it may change the content information so that the content information satisfies the predetermined condition.

[0141] For example, if a screen capture captured by the capture unit 214 does not satisfy a predetermined condition, the content information control unit 216 may change the content information, which is at least one of information about the virtual camera that photographs the object used to generate the screen capture (e.g., the position, orientation, angle of view, etc. of the virtual camera) and information about the object (e.g., the position, orientation, parameters, posture, etc. of the object), so that the content information satisfies the predetermined condition.

[0142] When there are multiple items to be changed, the content information control unit 216 may determine, based on a given priority, which item should be given priority for change (for example, among the parts that make up a character, the character's eyes may be given priority).

[0143] The content information control unit 216 may change the image (the image drawn in the frame buffer) when it determines that the predetermined condition is not satisfied. In other words, the image (the image drawn in the frame buffer) is also an example of content information. The content information control unit 216 may identify a portion of the image that does not satisfy the predetermined condition and change the image to an image that satisfies the predetermined condition. For example, the content information control unit 216 may change the image to an image that satisfies the predetermined condition using a given generation AI (image generation AI). Furthermore, the content information control unit 216 may change the image to an image that satisfies the predetermined condition by using another image.

[0144] The screen capture generation unit 217 may generate a screen capture based on the changed content information.

[0145] The screen capture generation unit 217 may generate a screen capture again based on the changed content information, that is, the screen capture generation unit 217 may redraw the screen capture.

[0146] The screen capture generation unit 217 may generate a screen capture based on the learning result by the learning unit 222.

[0147] The condition control unit 218 may set the predetermined condition based on an input operation by the user. The condition control unit 218 may also set the predetermined condition based on the user's usage history (for example, in the case of game content, the game play history). The condition control unit 218 may also set the predetermined condition based on content information. The condition control unit 218 may also further change the predetermined condition based on the user's evaluation of the screen capture generated by the screen capture generation unit 217.

[0148] The evaluation unit 219 allows the user to evaluate the screen capture generated by the screen capture generation unit 217 .

[0149] When changing content information, the parameter control unit 220 changes the consumption value consumed from the user's predetermined parameters according to the degree of change.

[0150] The communication control unit 221 performs processing for transmitting and receiving data to and from each server device 10. The communication control unit 221 also performs processing for storing data received from the server device 10 in the storage unit 270, processing for analyzing the received data, and other control processing related to the transmission and reception of data.

[0151] The communication control unit 221 may store and manage destination information (network information such as IP addresses and port numbers) of other terminal devices 20 and the server device 10 in the storage unit 270 or the information storage medium 280. The communication control unit 221 may then communicate with the server device 10 when receiving input information from the user to start communication.

[0152] In particular, the communication control unit 221 transmits the user's identification information and input information to the server device 10. The communication control unit 221 may also receive data (predetermined parameters of the user, the user's web page, a screen, etc.) from the server device 10.

[0153] The communication control unit 221 may transmit and receive data to and from the server device 10 at a predetermined cycle, or may transmit and receive data to and from the server device 10 when input information is received from the input unit 260. In particular, the communication control unit 220 of this embodiment performs processing to receive given screen information (for example, screen information, etc.) from the server device 10.

[0154] The learning unit 222 learns the content information at the specified timing.

[0155] The presentation unit 223 performs processing to present given information on the display unit 290. For example, the presentation unit 223 presents changed content information to the user. The presentation unit 223 presents the screen capture generated by the screen capture generation unit 217 to the user.

[0156] The image generation unit 230 performs drawing processing based on various processes (for example, content processing) performed by the processing unit 200, thereby generating an image, which is output to the display unit 290 by the display control unit 211. The image generated by the image generation unit 230 may be a so-called two-dimensional image or a so-called three-dimensional image. Note that the image generation unit 230 may perform all or part of the same processing as the image generation unit 130 of the server device 10.

[0157] The sound processing unit 240 performs sound processing based on the results of various processes performed in the processing unit 200, generates (plays) sounds such as background music, sound effects, or voices, and outputs them to the sound output unit 292. Note that the sound processing unit 240 may perform all or part of the same processes as the sound processing unit 140 of the server device 10.

[0158] In the case where the terminal device 20 of this embodiment provides content to a plurality of terminal devices 20 by P2P or the like, one terminal device 20 may act as a host and execute processing. In such a case, the terminal device 20 as the host may execute processing of each processing unit of the server device 10 (part or all of each processing unit).

[0159] Furthermore, the terminal device 20 of this embodiment may perform the same processing as the server device 10 and display a screen on the display unit of the terminal device 20.

[0160] [4] Overview The present embodiment relates to a generation system for generating a screen capture, for example, a generation system for generating a screen capture of a screen that provides content.

[0161] For example, on the terminal device 20 of the user PA, an input operation by the user PA may be performed to obtain a screen (e.g., a video streaming screen, a game screen, etc.) containing an image seen from a virtual camera in a virtual space (e.g., a game space, a metaverse) as a screen capture.

[0162] The object to be captured is an image seen from the virtual camera (e.g., an object placed in the virtual space). The image captured is an image of the frame buffer, which is created by overlaying an image layer of effects and user interface on a layer of information (such as information about the object) and drawing it in the frame buffer. For example, in the case of game content, the target of the screen capture is the screen on which a character is drawn in a virtual space.

[0163] 4 shows images G100 to G103 displayed for each frame (e.g., every 1 / 60 seconds) on the display unit 290 of the terminal device 20. For example, assume that the user PA obtains the screen capture image G100 at the timing T1. However, this image G100 may not be what the user PA wants because it does not show the character CA.

[0164] Furthermore, when the user PA acquires the screen capture image G101 at the timing T2, the character CA in this image G101 may be unclear and may not be what the user PA desires.

[0165] Furthermore, when the user PA acquires the screen capture image G102 at timing T3, the message MS is only partially displayed in this image G102, which may not be what the user PA desires.

[0166] Furthermore, when the user PA acquires the screen capture image G103 at timing T4, the image G103 may not display the character CB that the user wants, and may not be what the user PA wants.

[0167] Therefore, in this embodiment, the content information (for example, object placement information, camera position, etc.) based on the timing acquired by the user PA is changed, and a screen capture is generated again. In other words, the screen capture acquired by the user PA is corrected, and a screen capture that gives the user PA a sense of satisfaction is provided to the user PA.

[0168] As a result, in this embodiment, the best screen capture of the user PA's favorite characters, favorite scenes, etc. can be obtained without any failure.

[0169] In particular, in this embodiment, the user PA can set predetermined conditions that serve as criteria for determining whether the user PA is what they want. Furthermore, the predetermined conditions can be set by computer control based on the user PA's game play history, content status, etc.

[0170] The following description will mainly focus on an example in which content information is changed for a screen capture acquired based on an acquisition instruction from user PA, and the screen capture is generated again. In addition, in this embodiment, the processing of terminal device 20 of user PA will mainly be described, but some or all of the processing of terminal device 20 may be performed by server device 10.

[0171] [5] Screen capture description In this embodiment, "screen capture" includes both screenshots (still images) and screen recordings (video). For example, "screen capture" includes both still images and video recorded from the content displayed on the screen of a user's terminal device 20 (e.g., the display unit of a computer, smartphone, etc.).

[0172] Furthermore, in this embodiment, not only screen captures acquired based on user instructions, but also still images and videos generated by the generation system are treated as screen captures.

[0173] [6] Screen capture based on user input In this embodiment, an instruction to acquire a screen capture is accepted based on an input operation by a user PA. For example, in this embodiment, as shown in Fig. 4, when an instruction to acquire a screen capture is accepted at timing T1, the timing T1 is set as the instruction timing. Then, a screen capture at the instruction timing T1 is acquired.

[0174] For example, the terminal device 20 receives instruction timing T1 via the input unit 260 (for example, a given button input, touch input, etc.), and acquires a screen capture (still image) of the screen at instruction timing T1.

[0175] Furthermore, when the terminal device 20 receives an instruction timing T2 to start the video via the input unit 260, it starts taking a screen capture (video) of the screen from the instruction timing T2, and when it receives an instruction timing T3 to end the video, it ends taking a screen capture of the screen at the instruction timing T3.

[0176] In this embodiment, the acquired screen capture (still image, video, etc.) is stored in the screen capture storage unit 275 based on an input operation by the user PA, for example, in association with a unique capture ID.

[0177] In addition, this embodiment has a continuous screen capture function, for example, the image of each frame is stored in the screen capture storage unit 275 in a period before and after the timing T1 of the screen capture instruction.

[0178] 5, when a screen capture instruction timing T1 is received, each frame image is acquired from timing TS1, which is several seconds before the screen capture instruction timing T1, to timing TE1, which is several seconds after the instruction timing T1. That is, during the period TA from timing TS1 to timing TE1, each frame image G1 to G200 is stored in the screen capture storage unit 275.

[0179] The same applies to the case of capturing moving images. As shown in Fig. 6, when a command T2 to start a screen capture moving image is received, frame images are acquired from timing TS2, which is several seconds before the command T2 to start the screen capture, to timing TE2, which is several seconds after timing T3 to end the moving image. That is, during the period TB from timing TS2 to timing TE2, frame images G201 to G800 are stored in screen capture storage unit 275.

[0180] [7] Content Description In this embodiment, content includes entertainment content such as games, and image and video content delivered to users.

[0181] The content also includes communication services such as the metaverse and SNS. For example, the content may include services and content in the metaverse. The content may also include communication services in virtual spaces.

[0182] [8] Content information description "Content information" includes at least one of the following information: facial expression, position, orientation, size, shape, facial expression, posture, attributes, and change information of an object (e.g., a character) located in the virtual space, and virtual camera information (position (viewpoint), line of sight, and angle of view) of the virtual camera. Note that the virtual camera is a virtual camera that photographs the object.

[0183] The object is, for example, a subject (for example, a character).

[0184] The content information may include information related to the object (e.g., a character) (e.g., an item equipped by the character). The content information also includes model data and motion data of the character (e.g., kick motion data, punch motion data, etc.).

[0185] In a three-dimensional virtual space, the position of an object is represented by X, Y, and Z (depth) coordinate values. In a two-dimensional virtual space, the position of an object is represented by X and Y coordinate values.

[0186] The content information may also include information about the distance between objects. The content information may also include basic information about the user (attributes, billing information) and setting information for the content (character level, character type, relationships between characters). The content information may also include information about the auditory sense (sound data such as voice, background music, and sound effects).

[0187] Furthermore, when the terminal device 20 of this embodiment detects the position of the object and the position of the virtual camera, the content information may include the distance of the object from the virtual camera.

[0188] The content information may also include information such as other characters that appear around the subject or in the background, information such as transitions and switches of text and the entire screen, image pixels and transparency information (transparency, alpha value), image pixel values, and signals at the start and end of character display processing and audio playback processing.

[0189] Fig. 7 shows an example of drawing an image layer and a text layer in an overlapping manner in this embodiment. In this embodiment, as shown in Fig. 7, the content information may be an image GL or an image TL of text when the image TL (layer) is drawn in the frame buffer FB by overlapping the image GL (layer) of text on the image (layer) GL seen from the virtual camera.

[0190] The content information of this embodiment may also include background objects and effect objects in the virtual space.

[0191] In this embodiment, the content information is stored in the content information storage unit 273 of the terminal device 20. Note that the content information may also be stored in the content information storage unit 173 of the server device .

[0192] Furthermore, the content information stored in the content information storage unit 273 (or the content information storage unit 173) is not limited to content information on instruction timing.

[0193] For example, as shown in Fig. 5, in the case of an instruction to screen capture a still image, the content information stored in the content information storage unit 273 (or the content information storage unit 173) may include content information before the instruction timing T1 (for example, the period from TS1 to T1). It may also include content information for the period after the instruction timing T1 (for example, the period from T1 to TE1). It may also include content information for the period TA before and after the instruction timing T1 (for example, the period from TS1 to TE1).

[0194] For example, as shown in FIG. 6, in the case of an instruction to screen capture a moving image, the content information stored in the content information storage unit 273 (or the content information storage unit 173) includes the content information before the instruction timing T2 to start the moving image (for example, the period from TS1 to T3). It may also include content information for a period after the timing T3 of the instruction to end the moving image (for example, the period from T2 to TE2). It may also include content information for a period before the timing T2 of the instruction to start the moving image and a period after the timing T3 of the instruction to end the moving image (for example, the period from TS2 to TE2).

[0195] [9] Setting of specified conditions [9.1] User-defined conditions In this embodiment, predetermined conditions are set based on input operations by the user PA. For example, the user PA can set his / her own preferences.

[0196] For example, a predetermined condition may be that the virtual camera is directed toward the character CA1 based on an input operation by the user PA.

[0197] Alternatively, the predetermined condition may be that the color tone of the image is a predetermined color tone (for example, the average value of the RGB values ​​of each pixel is equal to or greater than a predetermined value (or equal to or less than a predetermined value)).

[0198] Also, the predetermined condition may be that a predetermined character (for example, a character that the user PA likes) is present in a screen capture. For example, the predetermined condition may be that the predetermined character is located within the field of view of the virtual space.

[0199] [9.2] Setting predetermined conditions based on the user's play history In this embodiment, a predetermined condition may be set based on the play history of the user PA. For example, the predetermined condition may be that a character frequently used by the user PA (for example, a character that has appeared a predetermined number of times or more in the play history of the user PA) is present in the screen capture.

[0200] Furthermore, a predetermined condition may be that a technique that is frequently used by the user PA (for example, a technique that has been used a predetermined number of times or more in the play history of the user PA) is present in the screen capture.

[0201] [9.3] Setting predetermined conditions based on content information In this embodiment, a predetermined condition may be set based on content information.

[0202] For example, the user PA may set a predetermined condition that the facial expression of the character that is the subject of the screen capture is a predetermined facial expression (e.g., a smile) of a character that the user PA likes (e.g., a character that has appeared a predetermined number of times or more in the user PA's play history).

[0203] Furthermore, for example, the user PA may set a predetermined condition that the posture of the character that is the subject of the screen capture is a predetermined posture (e.g., a posture with hands raised) of a character that the user PA likes (e.g., a character that has appeared a predetermined number of times or more in the user PA's play history).

[0204]

[10] Explanation of the process to determine whether a given condition is met In this embodiment, it is determined whether or not a screen capture (for example, an image or video) acquired by the user PA at the timing instructed by the user PA satisfies a predetermined condition.

[0205]

[11] Changes to content information In this embodiment, if it is determined that the screen capture (for example, an image or a video) acquired at the timing designated by the user PA does not satisfy a predetermined condition, Change content information.

[0206] [11.1] Changing model data In this embodiment, if it is determined that the screen capture (e.g., an image or video) acquired by the user PA at the timing of instruction does not satisfy specified conditions, the model data of the character (an example of content information) based on the timing of instruction may be changed.

[0207] 8 is a diagram illustrating an example of a change in model data in this embodiment. For example, suppose that an object OB1 drawn in the frame buffer FB of a screen capture captured by a user is in a state of holding out its hand, and it is determined that the object does not satisfy a predetermined condition (for example, the condition that the hand is outstretched). In such a case, the model data of the object OB1 in the image is changed to a state in which the hand is outstretched, as shown in FIG.

[0208] Then, an object OB1 is placed in the virtual space in a position that looks like it is sticking out its hand, and an image is generated that is seen from a virtual camera with the object OB1 as the subject. For example, the generated image is overlaid on a text layer and drawn (redrawn) in the frame buffer FB. In other words, the image drawn in the frame buffer FB is generated as a screen capture.

[0209] It should be noted that the content information is not limited to model data, and may be, for example, information about objects (characters, etc.) to be placed in the virtual space, as described above.

[0210] Furthermore, this may be realized not only in still images but also in videos. For example, a video may be generated by changing the motion data of the object OB1 from a punch to a kick, and a screen capture of the video may be generated.

[0211] [11.2] Changing the image in the framebuffer In this embodiment, if it is determined that the screen capture (for example, image or video) acquired at the timing designated by the user PA does not satisfy a predetermined condition, the image in the frame buffer based on the timing designated may be changed.

[0212] For example, a portion of the captured screen capture (e.g., an image drawn in the frame buffer FB1) that does not satisfy a predetermined condition may be identified, and the screen capture itself may be modified so that the portion satisfies the predetermined condition.

[0213] 9 is a diagram illustrating an example of changing an image drawn in a frame buffer in this embodiment. As shown in FIG. 9, suppose that an object OB1 drawn in a frame buffer FB1 of a screen capture captured by a user has its eyes closed and a sullen expression, and is determined not to satisfy a predetermined condition (for example, a smiling face). In such a case, an image is generated in which the eyes are changed to be open and the expression is changed to a slightly smiling face.

[0214] The change in content information may be a change related to image generation in the image generation unit 230. For example, it may be a change in pixel color data (RGB data), transparency (α value), texture, etc. Furthermore, it may be realized not only for still images but also for moving images.

[0215] Additionally, the screen capture captured by the user may be edited or modified by overlaying another image. For example, an image processing program may be used to automatically process the image drawn in the frame buffer FB1 shown in FIG.

[0216] Furthermore, a screen capture captured by a user may be modified by instructing the generation AI. When using an image generation AI to generate a new image that satisfies certain conditions, for example, a text-based instruction (prompt) describing the certain conditions is generated and sent to the image generation AI. Based on the instruction, the image generation AI generates an image that satisfies the specified certain conditions. Note that either the generated image or the captured image may be selected as the final captured image. Furthermore, if the captured image does not satisfy the certain conditions, the computer may determine the points that do not satisfy the conditions and input the points that do not satisfy the conditions and information about the captured image into the generation AI, causing the generation AI to generate an image that satisfies the conditions.

[0217] For example, in this embodiment, a text prompt that clearly states a predetermined condition, such as "Please change the image drawn in frame buffer FB1 to one that satisfies the condition of a smiling face," is generated for the image drawn in frame buffer FB1, and sent to the image generation AI. Based on the provided information, the image generation AI changes the image in frame buffer FB1 to an image that shows a smiling face and generates it. In this embodiment, the image generated by the image generation AI is received and saved as a new screen capture.

[0218] Furthermore, for example, if an image of an object raising its right hand is a predetermined condition, and if the right hand is not raised in a screen capture captured by a user, a screen capture may be generated that disguises whether the right hand is raised or not by overlaying another image (such as an effect or UI image) on the part of the right hand. Also, an instruction may be given to the image generation AI to modify the image so that the right hand is raised, and the image generation AI may generate a screen capture of the object raising its right hand.

[0219] Alternatively, instead of using an image generation AI, a plurality of images may be prepared in advance (stored in the storage unit 170) and replaced with an image that satisfies a predetermined condition.

[0220] Image generation AI is a technology that automatically generates a new image by giving instructions (e.g., text prompts). In this embodiment, instructions are sent from terminal device 20 to the image generation AI device, and an image generated by the image generation AI device is acquired (received). The acquired image is used as a new screen capture to be presented to user PA.

[0221] [11.3] Example of changing menu image In this embodiment, if it is determined that the screen capture (e.g., an image or video) acquired by the user PA at the specified timing does not satisfy a predetermined condition, the image data (an example of content information) based on the specified timing may be changed.

[0222] FIG. 10 is a diagram for explaining an example of changing an image drawn in the frame buffer in this embodiment.

[0223] For example, suppose that the menu image drawn in the frame buffer FB11 of a screen capture captured by a user does not have a star pattern and is determined not to satisfy a predetermined condition (for example, the condition that a star-patterned menu be displayed). In this case, the image is changed to an image with a star-patterned display object MK added, as shown in Fig. 10. In other words, the image drawn in the frame buffer FB12 is generated as a screen capture.

[0224] [11.4] Example of content information changes for still images In this embodiment, when the predetermined condition of the user PA is "the character CA exists," If the character CA does not exist in the screen capture G100 of the still image acquired based on the instruction timing T1 from the user PA, the character CA is changed to be placed in the virtual space at the instruction timing T1. For example, at the instruction timing T1, the character CA is placed within the field of view of the virtual space so that it faces the virtual camera.

[0225] [11.5] Example of changing content information for a video In this embodiment, if the specified condition of the user PA is that "the attack technique is a kick," and the screen capture MV of the video acquired based on the period from the user PA's instruction timing T2 to T3 is a punch technique and not a "kick," the character's technique is changed to a kick.

[0226] When a character's technique is changed from a punch to a kick, the visual images (affects and UI (user interface)) and auditory information (lines, sound effects, BGM (background music)) may also be changed accordingly.

[0227] [11.6] Priorities In this embodiment, when there are multiple items to be changed, the item to be changed with priority may be determined based on a given priority. For example, the item to be changed with priority may be set in advance based on operation information of the user PA.

[0228]

[12] Consumption of specified parameters In this embodiment, the content information may be changed on the condition that a predetermined parameter (for example, points) is consumed from the user PA. The server device 10 of this embodiment stores points in the user information storage unit 174 in advance in association with a user ID for each user. Furthermore, the terminal device 20 of the user PA stores the points of the user PA in the user information storage unit 274.

[0229] For example, if the points of user PA are "100" and the content information for the screen capture of user PA is to be changed, the consumption value (e.g., "10") is consumed (subtracted) from the points associated with user PA, and the content information is changed.

[0230] In this embodiment, when the content information of the user PA is changed, the consumption value consumed by the user PA may be changed according to the degree of the change.

[0231] For example, the greater the number of changes, the higher the consumption value. Furthermore, the larger the area of ​​the changed area in the screen capture image, the higher the consumption value. Furthermore, if the change is to a main subject, such as a character, the consumption value associated with the change may be higher than the consumption value associated with a change to the background.

[0232]

[13] Generating a screen capture In this embodiment, when content information is changed, a screen capture is generated based on the changed content information.

[0233] [13.1] Still Image Generation In this embodiment, when the predetermined condition of the user PA is "character CA exists," if character CA does not exist in the screen capture G100 of the still image acquired based on the instruction timing T1 of the user PA, character CA is placed in the virtual space at the instruction timing T1, and an image G100-A seen from the virtual camera is generated. .

[0234] [13.2] Video Generation In this embodiment, when the predetermined condition of the user PA is that "the attack technique is a kick," if a screen capture MV of a moving image acquired based on the period of instruction timing T2 to T3 of the user PA is a screen capture of a punch technique, the character's technique is changed to a kick and a screen capture MV-A is generated.

[0235] For example, in this embodiment, character motion data (movement information) is stored in advance in the content information storage unit 273. Then, when the character's movement is changed to "kick," the completion point of the kick motion data is set as the end point of the screen capture MV-A.

[0236]

[14] Presentation In this embodiment, a generated screen capture is presented to the user PA based on the changed content information.

[0237] [14.1] Example 1 11 is an example of a screen presented on the display unit 290 of the terminal device 20 of the user PA. In this embodiment, the generated screen capture G100-A is displayed on the terminal device 20 of the user PA, as shown in FIG.

[0238] In this embodiment, the changed content information may be presented to the user PA as shown in Fig. 11. For example, the generated screen capture G100-A may be presented to the user with a message MA saying "The character CA has been positioned so that it will be displayed."

[0239] In this embodiment, the screen capture G100 captured based on an instruction from the user PA and the generated screen capture G100-A may be presented. Also, based on an operation input from the user PA, a selection of either the screen capture G100 or the screen capture G100-A may be accepted and stored in the screen capture unit 275.

[0240] [14.2] Example 2 In this embodiment, if the predetermined condition of the user PA is "the attack technique is a kick," and the screen capture MV of the video acquired based on the instruction of the user PA is a screen capture of a punch technique, the character's technique is changed to a kick, and if a screen capture MV-A is generated, the screen capture MV-A is presented. Note that presenting the screen capture MV-A may also mean presenting a thumbnail for playing the screen capture MV-A.

[0241] Furthermore, for example, if the attacking motion is changed from a punch to a kick, the generated screen capture MV-A is presented to the user with the information "the attacking motion has been changed from a punch to a kick" added to it.

[0242] In this embodiment, the screen capture MV captured based on an instruction from the user PA and the generated screen capture MV-A may be presented. Also, based on an operation input from the user PA, a selection of either the screen capture MV or the screen capture MV-A may be accepted and stored in the screen capture unit 275.

[0243]

[15] Evaluation [15.1] Acceptance of evaluations In this embodiment, the generated screen capture GA may be evaluated based on the changed content information.

[0244] In this embodiment, the evaluation is performed based on the operation input of the user PA. For example, as shown in FIG. 5, an evaluation screen is presented in which the user PA can select either a "good" instruction 51 or a "bad" instruction 52, and the user PA is allowed to input "good" or "bad." The evaluation method is not limited to a binary selection. For example, the user PA may input a score.

[0245] Also, if there are multiple screen captures, each screen capture may be assigned a rank.

[0246] Alternatively, multiple evaluation items may be set for one screen capture, and evaluations may be made for each evaluation item (e.g., evaluations of good or bad, scores, rankings, etc.) Evaluation items include, for example, the character's facial expression, the position, orientation, and angle of view of the virtual camera.

[0247] [15.2] Changes to prescribed conditions based on evaluation In this embodiment, the predetermined conditions may be changed based on the evaluation of the screen capture GA.

[0248] For example, if the evaluation of the screen capture GA by the user PA is a predetermined evaluation (for example, "bad" or the score is 50 points or less), the predetermined condition is changed to another content.

[0249] [15.3] Modifying content information based on ratings In this embodiment, the content information may be further changed based on the evaluation of the screen capture GA, thereby making it possible to provide a screen capture that satisfies the user PA.

[0250] For example, if the evaluation of the screen capture GA of the user PA is "bad," the content information is changed to another one.

[0251] In order to reflect the user's evaluation, consuming a predetermined value from a predetermined parameter may be set as a condition for changing the content information.

[0252] [15.4] Generating screen captures based on changed content information In this embodiment, if content information is changed based on evaluation of a generated screen capture (e.g., screen capture G100-A), a screen capture (e.g., screen capture G100-B) may be generated again based on the changed content information. For example, in this embodiment, the generated screen capture (e.g., screen capture G100-B) may be presented again.

[0253]

[16] Learning function [16.1] Learning content information at instruction timing In this embodiment, content information at the timing of the instruction may be learned, and a screen capture may be generated based on the learning result.

[0254] For example, in this embodiment, when information regarding the instruction timing is input, content information Then, information about the timing of the instruction is input and content information is estimated. A screen capture is then generated using the content information based on the estimation results.

[0255] "Information related to instruction timing" refers to information about the status of the content (e.g., the game status) at the time of the instruction. For example, "information related to instruction timing" refers to at least one of information about whether a technique has been executed, information about whether a character has posed, information about whether a user's comment is displayed in a video stream, and information about whether a character has responded to a user's input. The data set of information related to instruction timing and content information is stored in the learning memory unit 278.

[0256] The learning model storage unit 279 stores a learned model that has been learned based on a data set of information related to instruction timing and content information.

[0257] [16.2] Learning between given conditions and content information In this embodiment, predetermined conditions set for a user and content information may be learned as a data set. Then, using the learned learning model, predetermined conditions of the user PA are input and content information is estimated. Then, a screen capture may be generated based on the estimated content information. The generated screen capture may be presented to the user PA.

[0258] For example, the learning data may be predetermined conditions and content information corresponding to the user PA, or predetermined conditions and content information corresponding to a user other than the user PA (for example, the user PB).

[0259] The predetermined conditions and content information are stored as a data set in the learning storage unit 278.

[0260] The learning model storage unit 279 stores a learned model that has been learned based on a data set of predetermined conditions and content information.

[0261] [16.3] Details of the learning function The process of generating a learning model is as follows. Here, we will explain it using an example that uses a common machine learning or deep learning framework.

[0262] Deep learning frameworks are program libraries and tools for efficiently designing, training, and evaluating deep learning models. These frameworks provide high-level APIs that make it easy to build neural networks, manage the training process, optimize computations, and implement complex architectures such as multi-layer perceptrons, convolutional neural networks (CNNs), and recurrent neural networks (RNNs).

[0263] The purpose of the learning model is to obtain output information from input information. For example, a learning model that estimates content information (an example of output information) from information about instruction timing (an example of input information) and a learning model that estimates content information (an example of output information) from predetermined conditions (an example of input information) are generated.

[0264] The input information can be in any format, such as time series data, images, text, etc. The output information can be in a format appropriate for the purpose, such as predicted values, classification labels, or continuous values.

[0265] Next, data collection and pre-processing are performed. That is, as data collection, a data set including information on instruction timing and content information corresponding thereto is collected.

[0266] Preprocessing involves transforming the data into a format that is easier for the model to process, such as normalization, handling missing values, encoding, etc.

[0267] Next, select a model (algorithm). For example, select a machine learning or deep learning model appropriate for the type (regression, classification, clustering, etc.). If using deep learning, select an appropriate network architecture (CNN, RNN, LSTM, Transformer, etc.).

[0268] CNN stands for Convolutional Neural Network and is a deep learning model used for image recognition and other purposes.

[0269] RNN stands for Recurrent Neural Network and is a deep learning model that learns patterns in time series data.

[0270] LSTM stands for Long Short Team Memory and is a type of RNN that can capture long-term dependencies.

[0271] The Transformer is a deep learning model that processes sequence data using a self-attention mechanism.

[0272] Next, as part of the learning process, data splitting is performed, i.e., the dataset is divided into a training set, a validation set, and a test set.

[0273] Then, the model is trained, i.e., trained using the training data. At this stage, you may define the loss function, optimization algorithm, batch size, number of epochs, etc.

[0274] Additionally, hyperparameter tuning may be performed, i.e., hyperparameters (such as learning rate, number of hidden layers, number of nodes, etc.) may be adjusted to optimize the performance of the model.

[0275] Performance evaluation may also be performed. The validation set may be used to evaluate the model's performance and check for overfitting or underfitting. For example, model tuning (model optimization) may be performed. In other words, the model structure may be adjusted as necessary and retrained.

[0276] In this embodiment, the trained model is stored in the trained model storage unit 279 and converted into a format that can be used in actual applications.

[0277] A model can be thought of as a function. In machine learning and deep learning, a "model" is a collection of mathematical functions that make predictions or inferences from input data. This function expresses the relationship between inputs (features) and outputs (predictions or class labels) and captures patterns and structures learned from the data. During the training process, the parameters of this function (weights and biases) are adjusted to find the best fit to the data.

[0278]

[17] Screen capture of video In this embodiment, when the screen capture is a video, at least the duration of the video, the start time of the video, the end time of the video, and the playback data of the audio (volume and balance) are recorded. You may want to change one of them.

[0279] In this embodiment, the period (length) of the moving image may be changed to be longer or shorter than the period actually instructed by the user in order to generate a moving image as desired by the user.

[0280] For example, if user PA captures a scene of the meeting between characters CA and CB for a period of 10 seconds, but the scene is actually 20 seconds long, the duration of the video is changed so that it is 20 seconds, even though the period specified by the user is 10 seconds.

[0281] Also, for example, a fade-in effect may be added at the start of the video, and a fade-out effect may be added at the end of the video.

[0282] Also, changing the volume of the audio playback data for a video means increasing the volume (amplitude) of the audio (making it easier to hear), while lowering the volume (amplitude) of the audio means making it smaller (quieter).

[0283] Also, changing the balance of the audio playback data of a moving image means changing the volume ratio of the background audio and the character audio, for example.

[0284]

[18] Flowchart The processing flow of this embodiment will be described with reference to Fig. 12. First, a screen capture instruction is accepted based on a user's input operation (step S1). At the timing of the instruction, the screen is captured (step S2). It is determined whether the screen capture satisfies a predetermined condition (step S3).

[0285] If the screen capture does not satisfy the predetermined conditions (N in step S3), the content information based on the instruction timing is changed (step S4).

[0286] Then, a screen capture is generated based on the changed content information (step S5).

[0287] If the screen capture satisfies the predetermined condition (Y in step S3), the process ends.

[0288]

[19] Other examples Another example of changing content information and generating a screen capture in this embodiment will be described. [19.1] Example 1 In this embodiment, when the predetermined condition of the user PA is "display character CA," for example, assume that the instruction timing of the user PA is T2, as shown in FIG. 4. Then, in a screen capture G101 of a still image acquired based on the instruction timing T2 of the user PA, if character CA is present but is displayed transparently through transparency processing, the alpha value (an example of content information) of the generated image of character CA is changed to be completely opaque at the instruction timing T2. Then, an image including character CA is generated by changing the alpha value. Then, in this embodiment, the generated image is presented.

[0289] [19.2] Example 2 In this embodiment, when the predetermined condition of user PA is "display a message from character CA," for example, assume that user PA's instruction timing is T3, as shown in FIG. 4. Then, if character CA's message is displayed only partially (for example, "Hello. I am") in a still image screen capture G102 acquired based on user PA's instruction timing T3, the message content (an example of content information) is changed at instruction timing T3 so that the message content is displayed to the end. Then, an image is generated containing character CA's message with the changed message content. Then, in this embodiment, the generated image is presented.

[0290] [19.3] Example 3 In this embodiment, if the predetermined condition of the user PA is "the character must be smiling," and the character CA is not smiling in a screen capture of a still image acquired based on the instruction timing Tx1 of the user PA, the facial expression of the character CA (an example of content information) is changed to a smiling facial expression at the instruction timing Tx1. Then, an image of the character CA after the facial expression change is generated. Then, in this embodiment, the generated image is presented.

[0291] [19.4] Example 4 In this embodiment, when the predetermined condition of the user PA is "character conversation," if the conversation of the character CA starts midway in a screen capture of a video acquired based on the instruction timings Tx2 to Tx3 of the user PA, the start time of the video (an example of content information) is changed. For example, a video is generated that starts at timing TSx before the instruction timing Tx2 and ends at Tx3. Then, in this embodiment, the generated video is presented.

[0292] Furthermore, if the conversation of character CA ends midway in the screen capture, the end point of the video (an example of content information) is changed. For example, a video is generated that ends at timing TEx, which is later than timing Tx3 instructed to end the video. In other words, a video is generated for the period from timing Tsx to TEx. Then, in this embodiment, the generated video is presented. Note that the audio data is also changed to match the generated video.

[0293]

[20] Best Screen Capture Selection [20.1] User Selection In this embodiment, the best screen capture may be selected from each frame image of a predetermined period (for example, the period TA shown in FIG. 5) stored in the screen capture storage unit 275 based on an input operation by the user PA.

[0294] In addition, in this embodiment, the start and end points of the best moving image period may be selectable from among the frame images of a predetermined period (for example, the period TB shown in FIG. 6) stored in the screen capture memory unit 275 based on the input operation of the user PA.

[0295] [20.2] Condition-Based Selection In this embodiment, from each frame image of a predetermined period (for example, the period TA shown in FIG. 5) stored in the screen capture memory unit 275, signals indicating the time when the image becomes opaque in the transparency processing (transparency processing), the end of the character display processing, and the start and end of the audio playback processing may be obtained from the program, and that timing may be selected as the best screen capture.

[0296] In this embodiment, the difference between each pixel of the previous and next images is calculated from each frame image of a predetermined period (for example, the period TA shown in FIG. 5) stored in the screen capture storage unit 275. The difference between the two images may be calculated and the best screen capture may be selected based on the difference.

[0297] In addition, in this embodiment, AI for facial image recognition may be used to select an image in which the eyes are fully open as the best screen capture.

[0298]

[21] Handling of screen captures that do not meet the specified conditions In this embodiment, a screen capture G100 that is determined not to satisfy a predetermined condition (for example, the condition that a character exists) may be stored in the screen capture storage unit 275. Furthermore, content information may be changed so as to satisfy the predetermined condition, and the screen capture G100-A generated based on the content information may be associated with the content information and stored in the screen capture storage unit 275. In this way, the user PA can enjoy viewing the screen capture G100, for example, like a blooper reel.

[0299]

[22] Metaverse The metaverse refers to a virtual space on the Internet where users can communicate with each other. The virtual space in the metaverse can be accessed using devices such as VR (virtual reality) goggles or AR (augmented reality) glasses. Users can create their own avatars and enjoy a variety of services and content. Users may also be able to play games in the metaverse.

[0300]

[23] Augmented reality The image generated in this embodiment may be an image in which an image of the actual real world is superimposed on an image of an object seen from a virtual space using augmented reality (AR) technology.

[0301] For example, in this embodiment, if the screen capture captured at the instruction timing does not satisfy specified conditions, the content information based on the instruction timing may be changed, an image Gy of the object visible from the virtual space may be generated based on the changed content information, and a new screen capture may be generated using an image (moving image) in which the image Gy is superimposed on an image ARy of the real world.

[0302]

[24] Processing of server equipment In the present embodiment, an example has been described in which content information is changed and a screen capture is generated mainly in the terminal device 20, but content information may be changed and a screen capture may be generated in the server device 10. In such a case, the server device 10 may have the same functions as the terminal device 20, such as a function for setting predetermined conditions, an evaluation function, and a learning function.

[0303]

[25] Other The present invention is not limited to the above-described embodiments, and various modifications are possible. For example, terms cited in the specification or drawings as broadly defined or synonymous terms can be replaced with broadly defined or synonymous terms in other descriptions in the specification or drawings.

[0304] The present invention includes configurations that are substantially the same as the configurations described in the embodiments (for example, configurations with the same functions, methods, and results, or configurations with the same purpose and effects). The present invention also includes configurations in which non-essential parts of the configurations described in the embodiments are replaced. The present invention also includes configurations that have the same effects as the configurations described in the embodiments or that can achieve the same purpose. The present invention also includes configurations in which publicly known technology is added to the configurations described in the embodiments. .

[0305] Although the embodiments of the present invention have been described in detail as above, it will be readily apparent to those skilled in the art that many modifications can be made without substantially departing from the novel features and effects of the present invention. Therefore, all such modifications are intended to be included within the scope of the present invention. [Explanation of symbols]

[0306] 10 Server device, 20, 20A, 20B, 20C Terminal device, 100 processing unit, 111 content control unit, 112 display control unit, 113 reception unit, 120 presentation unit, 121 communication control unit, 122 management unit, 130 image generation unit, 140 sound processing unit, 160 input unit, 170 memory unit, 171 main memory unit, 172 image buffer, 173 content information storage unit, 174 user information storage unit, 180 Information storage media, 196 Communications Department, 200 processing unit, 211 content control unit, 212 display control unit, 213 reception unit, 214 capture unit, 215 determination unit, 216 content information control unit, 217 Screen capture generation unit, 218 Condition control unit, 219 Evaluation unit, 220 parameter control unit, 221 communication control unit, 222 learning unit, 223 presentation unit, 230 image generation unit, 240 sound processing unit, 260 input unit, 262 detection unit, 270 storage unit, 271 main memory unit, 272 image buffer, 273 content information memory unit, 274 User information storage unit, 275 Screen capture storage unit, 276 Evaluation information memory unit, 278 Learning memory unit, 279 Learning model memory unit, 280 Information storage medium, 290 Display section, 292 Sound output section, 296 Communication section

Claims

1. 1. A generation system for generating a screen capture of a screen providing content, comprising: a receiving unit that receives a screen capture instruction based on an input operation by a user; a capture unit that captures the screen at the instruction timing when the instruction is received; a determination unit that determines whether the screen capture captured by the capture unit satisfies a predetermined condition; a content information control unit that changes content information based on the instruction timing when it is determined that the predetermined condition is not satisfied; and a screen capture generation unit that generates a screen capture based on the changed content information.

2. In claim 1, A generation system further comprising a condition control unit that sets the predetermined conditions based on an input operation by a user.

3. In claim 1 or 2, A generation system further comprising a condition control unit that sets the predetermined conditions based on a user's usage history.

4. In claim 1 or 2, A generation system further comprising a condition control unit that sets the predetermined conditions based on content information.

5. In claim 1 or 2, The content information control unit A generation system characterized in that the content information is changed when a predetermined parameter of a user is consumed.

6. In claim 1 or 2, The generation system further comprises a parameter control unit that, when the content information is changed, changes a consumption value to be consumed from a predetermined parameter of the user in accordance with the degree of the change.

7. In claim 1 or 2, The generation system further comprises a presentation unit that presents the changed content information to a user.

8. In claim 1 or 2, a presentation unit that presents the screen capture generated by the screen capture generation unit to the user; The generation system further comprises an evaluation unit that allows the user to evaluate the screen capture generated by the screen capture generation unit.

9. In claim 1 or 2, The generation system further comprises a condition control unit that further changes the predetermined condition based on the user's evaluation of the screen capture generated by the screen capture generation unit.

10. In claim 1 or 2, The content information control unit further modifying the content information based on the user's evaluation of the screen capture generated by the screen capture generator; The screen capture generation unit A generation system that generates a screen capture again based on the changed content information.

11. In claim 1 or 2, a learning unit that learns content information at the instruction timing, The screen capture generation unit A generation system that generates the screen capture based on the learning results of the learning unit.

12. In claim 1 or 2, The content information control unit A generation system characterized in that, when the screen capture is a video, at least one of the duration of the video, the start point of the video, the end point of the video, and the audio playback data of the video is changed.

13. In claim 1 or 2, The content information control unit If the screen capture captured by the capture unit does not satisfy the predetermined condition, content information, which is at least one of information about the virtual camera that photographs the object and information about the object used to generate the screen capture, is changed so as to satisfy the predetermined condition; The screen capture generation unit A generation system for redrawing a screen capture.

14. In claim 13, The content information control unit A generation system characterized in that, when there are multiple items to be changed, an item to be changed with priority is determined based on a given priority.

15. In claim 1, The determination unit determining whether the screen capture captured by the capture unit is an image drawn in a frame buffer and whether the image satisfies the predetermined condition; The content information control unit If the predetermined condition is not satisfied, the image is modified so that the predetermined condition is satisfied.

16. In claim 15, The content information control unit A generation system that identifies a portion of the image that does not satisfy the predetermined condition, and changes the image to an image that satisfies the predetermined condition.

17. In claim 15 or 16, The content information control unit A generation system that uses a given generation AI to change the image into an image that meets the specified conditions.

18. In claim 15 or 16, The content information control unit A generation system that changes the image into an image that satisfies the predetermined conditions using another image.

19. A program for generating a screen capture of a screen providing content, a receiving unit that receives a screen capture instruction based on an input operation by a user; a capture unit that captures the screen at the instruction timing when the instruction is received; a determination unit that determines whether the screen capture captured by the capture unit satisfies a predetermined condition; a content information control unit that changes content information based on the instruction timing when it is determined that the predetermined condition is not satisfied; A program that causes a computer to function as a screen capture generation unit that generates a screen capture based on changed content information.

20. An information processing device that generates a screen capture of a screen that provides content, a receiving unit that receives a screen capture instruction based on an input operation by a user; a capture unit that captures the screen at the instruction timing when the instruction is received; a determination unit that determines whether the screen capture captured by the capture unit satisfies a predetermined condition; a content information control unit that changes content information based on the instruction timing when it is determined that the predetermined condition is not satisfied; and a screen capture generation unit that generates a screen capture based on the changed content information.

Images