Information processing device, information processing method, program, and storage medium

Abstract

To provide an information processing device, an information processing method, a program, and a storage medium capable of reducing a processing load and applying a post effect different in each optional screen region designated by a producer.SOLUTION: In one embodiment of the present invention, a mask region corresponding to each object to which an effect is applied is drawn in a mask buffer S41, a corresponding object is drawn in a camera butter S42, pixels in a whole screen of a predetermined image drawn in the camera buffer are scanned using the mask region S43, the color of a mask region for each of the scanned pixels is determined S44a-44c, an effect to a corresponding pixel is not applied when the color is not associated with an effect to be applied as a result of the determination, and an effect according to the color determined in each pixel is applied when the color is associated with the effect to be applied as a result of the determination S45a-45c.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, a program, and a storage medium. [Background technology]

[0002] In real-time 3D (Three Dimensional) graphics for games and the like, the general flow of forward rendering is as follows: (i) coordinate transformation (3D coordinates → 2D coordinates), (ii) object drawing, (iii) post-effects.

[0003] The production of cut scenes that appear in games often requires the creation of special post-effects that make full use of characters and VFX (Visual Effects), resulting in a huge amount of production work for artists. Note that post-effects generally refer to dramatic effects that are applied to the entire screen after the screen rendering is complete (see Non-Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Non-Patent Document 1] Broadcast in April 2022! Unity Japan YouTube Summary<https: / / note.com / unityjapan / n / nedccf7c425dd> Summary of the Invention [Problem to be solved by the invention]

[0005] In conventional renderers, post-effects are applied uniformly to the entire screen. Therefore, for example, in a scene in which background (object category (type) is background), characters (object category is characters), and VFX (object category is VFX) are depicted as objects, normal post-effect methods cannot apply different effects to each object category.

[0006] In contrast, according to a method of applying a post-effect every time an object is drawn, a different effect can be applied to each category of object. That is, a layer is set for each category for which post-processing is to be separated, an object is drawn for each category of object, the pixels of the entire screen are scanned for each layer to apply the post-effect corresponding to the corresponding category, and the layers to which the above-mentioned respective post-effects are applied are superimposed based on the depth of each object to composite each object. For example, if the object categories are background, character, and VFX, an object (background) is drawn on a layer for the background category, the pixels of the entire screen are scanned to apply the post-effect for the background, then an object (character) is drawn on a layer for the character, the pixels of the entire screen are scanned to apply the post-effect for the character, then an object (VFX) is drawn on a layer for VFX, the pixels of the entire screen are scanned to apply the post-effect for VFX, and composite processing (which object should be brought to the front) is performed based on the depth of each object.

[0007] This method requires complex compositing based on the depth of the object, and as the number of object categories increases, the number of pixel scans across the screen, which is a heavy load for information processing devices such as computers, also increases, making this method unsuitable for real-time applications such as games.

[0008] The present invention has been made in consideration of such problems, and its object is to provide an information processing device, information processing method, program, and storage medium that can reduce the processing load and apply different post-effects (design effects to the entire screen) to each arbitrary screen area (each category of object) specified by the creator. [Means for solving the problem]

[0009] In order to achieve such an object, a first aspect of the present invention is an information processing device capable of applying an effect to at least one of objects included in a specified image when generating the image in a game, comprising: means for drawing a mask area corresponding to each of the objects to which the effect is to be applied in a mask buffer, and means for drawing the corresponding object in a camera buffer to become an image to be output; means for scanning the pixels of the entire screen of the specified image drawn in the camera buffer, each of the colors being associated with the effect to be applied, using the mask data as a mask, and applying the corresponding effect to pixels of the scanned pixels to which the effect is to be applied, means for determining the color of the mask area for each scanned pixel, and if the color is determined to be not associated with the effect to be applied, no effect is applied to the corresponding pixel, and if the color is determined to be associated with the effect to be applied, means for applying an effect according to the color determined for each pixel.

[0010] A second aspect of the present invention is an information processing method capable of applying an effect to at least one of objects included in a specified image when generating an image in a game, comprising the steps of: drawing a mask area corresponding to each of the objects to which the effect is to be applied in a mask buffer, wherein the mask area is drawn in a color associated with each category of the objects to which the effect is to be applied to generate mask data; drawing the corresponding objects in a camera buffer to become the image to be output; and a means for scanning the pixels of the entire screen of the specified image drawn in the camera buffer, each of which is associated with the effect to be applied, using the mask data as a mask, and applying the corresponding effect to pixels of the scanned pixels to which the effect is to be applied, wherein the color of the mask area for each scanned pixel is determined, and if the color is determined to be not associated with the effect to be applied, the effect is not applied to the corresponding pixel, and if the color is determined to be associated with the effect to be applied, the effect is applied according to the color determined for each pixel.

[0011] A third aspect of the present invention is a program that causes a computer to function as an information processing device capable of applying an effect to at least one of objects included in an image when generating a specified image in a game, the information processing device comprising: means for drawing a mask area corresponding to each of the objects to which the effect is to be applied in a mask buffer, the mask area being drawn in a color associated with each category of the objects to which the effect is to be applied to generate mask data; means for drawing the corresponding object in a camera buffer to become the image to be output; means for scanning the pixels of the entire screen of the specified image drawn in the camera buffer, each of the colors being associated with the effect to be applied, using the mask data as a mask, and applying the corresponding effect to pixels of the scanned pixels to which the effect is to be applied, the means for determining the color of the mask area for each scanned pixel, and if the color is determined to be not associated with the effect to be applied, the effect is not applied to the corresponding pixel, and if the color is determined to be associated with the effect to be applied, the means for applying an effect corresponding to the color determined for each pixel.

[0012] A fourth aspect of the present invention is a computer-readable storage medium storing the program according to the third aspect. Effect of the Invention

[0013] According to the present invention, it is possible to reduce the processing load and apply different post-effects (design effects to the entire screen) to each arbitrary screen area (each object category) designated by the creator. [Brief description of the drawings]

[0014] [Figure 1] 1 is a diagram showing a schematic configuration of an information processing apparatus according to an embodiment of the present invention; [Diagram 2] FIG. 2 illustrates an image of a game according to one embodiment of the present invention. [Diagram 3] FIG. 3 illustrates a mask created for the image of FIG. 2 according to an embodiment of the present invention. [Figure 4] 10 is a flowchart showing a processing procedure for applying an effect to each object according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings, but the present invention is not limited to this embodiment. In the drawings described below, parts having the same functions are given the same reference numerals, and repeated explanations of those parts may be omitted.

[0016] One embodiment of the present invention involves, when generating a specific image during a game, attaching a tag for specifying an area (mask) to an object to which an effect is to be applied in advance, and then applying the desired effect to each object in the image. Using the tag as a reference for each frame of processing, an area (mask) corresponding to the object is generated in a mask buffer, and the corresponding effect is applied to each object using the mask.

[0017] In the present invention, an "object" refers to something that is placed in each image in the game, such as a background, a character, a VRX, a weapon, an object that is not classified as a background (such as a motorcycle on which a certain character appears), etc. Note that the background may also include objects that are classified as background (such as trees, buildings, roads, etc.).

[0018] 1 is a diagram showing a schematic configuration of an information processing device 100 according to this embodiment. The information processing device 100 is a home-use stationary game machine, but may be other game machines such as a gaming PC (Personal Computer), an arcade game machine, a portable game machine, etc. It may also be a server, a PC, etc. The information processing device 100 includes a CPU (Central Processing Unit) 101, a GPU (Graphics Processing Unit) 102, a storage 103, a memory 104, an output interface (I / F) 105, a reader interface (I / F) 106, a reader 107, an input interface (I / F) 108, and a communication module 109. A display unit 110 such as a display is connected to the information processing device 100 via the output I / F 105, and an input unit 111 such as a mouse, keyboard, or controller is connected via the input I / F 108.

[0019] The CPU 101 executes various processing operations such as various calculations, controls, and determinations, and functions to execute various processes, for example, by loading a predetermined program from the storage 103 into the memory 104 and executing a series of instructions included in the program. The GPU 102 is a processor for performing image processing, and draws an image in response to a predetermined instruction from the CPU 101 and causes the display unit 110 to display the image.

[0020] The storage 103 stores programs (for example, the programs shown in FIG. 4) and various data to be read and executed by the CPU 101. The memory 104 temporarily holds the data and programs. The memory 104 is a volatile memory that temporarily stores data being processed by the CPU 101 and the GPU 102, input data, and further data (such as images) read by the reader 106 (a mask buffer, a camera buffer, etc., described later, are formed in the memory 104), and is, for example, a RAM (Random Access Memory).

[0021] The output I / F 105 displays an image on the display unit 110 according to instructions from the CPU 101 and the GPU 102. The reader 107 is provided in the information processing device 100 and is configured to be able to mount an external storage medium such as a memory card or a DVD (Digital Versatile Disc) storing various data, programs, etc., and reads predetermined data from the mounted external storage medium. The reader I / F 106 reads data from the external storage medium mounted in the reader 106 according to instructions from the CPU 101. The input I / F 108 accepts an input signal related to an operation input by a user through the input unit 111. The communication module 109 is connected to a communication network and communicates with other computers connected to the communication network. This communication may be wired or wireless. A so-called online game may be executed through the communication module 109.

[0022] In the following, in this embodiment, a case will be described in which there are three object categories: character, VFX, and background. In this embodiment, a mask corresponding to each object to which an effect is to be applied is generated for one image of a game (an image corresponding to each frame) as exemplified in Fig. 2. In Fig. 2, a character 201 is placed in an image 200, and a VFX of lightning directed at the character 201 is placed. In Fig. 2, the portion other than the character 201 and the VFX 202 is a background 203. Note that the background 203 includes a road 203a (i.e., the road 203a is also part of the background 203).

[0023] In this embodiment, for example, for an image as shown in Fig. 2, a mask as shown in Fig. 3 is generated in a mask buffer. In Fig. 3, mask data 300 corresponding to an image 200 includes a mask area (first area) 301 corresponding to a character 201, a mask area (second area) 302 corresponding to a VFX 202, and a mask area (third area) 303 corresponding to a background 203. In Fig. 2, since a road 203a is included in the background 203, there is no mask area in the shape of the road 203a in the mask data 300. In this embodiment, a different color is assigned to the corresponding mask area for each object category.

[0024] In this embodiment, three monochrome colors R (Red), G (Green), and B (Blue) are used as colors to be assigned to the mask regions. That is, green (G) is assigned to the mask region corresponding to the character object, blue (B) is assigned to the mask region corresponding to the VFX object, and red (R) is assigned to the mask region corresponding to the background object. Therefore, each pixel of the mask region 301 corresponding to the character has green color information, each pixel of the mask region 302 corresponding to the VFX has blue color information, and each pixel of the mask region 303 corresponding to the background has red color information.

[0025] In this embodiment, the mask process is not performed on objects to which no effect is applied. Therefore, for example, in the image 200, if no effect is applied to the VFX 202, the second region 302 in the mask data 300 in Fig. 3 remains in the default color (e.g., black) of the mask buffer.

[0026] In this embodiment, as described below, using the mask data 300 created in this manner, a first effect is applied to the character 201, a second effect different from the first effect is applied to the VFX 202, and a third effect different from the first and second effects is applied to the background 203.

[0027] Fig. 4 is a flowchart showing the processing procedure for applying an effect to each object according to this embodiment. The method for applying an effect shown in Fig. 4 is executed during the process of applying an effect. Alternatively, it may be executed for each rendering frame (i.e., it may be executed for each frame constituting a predetermined game). This process is performed by the CPU 101 reading the program shown in Fig. 4 stored in the storage 103 and executing the program.

[0028] In this embodiment, a tag is attached in advance to each object appearing in a given game to identify the category (type) of the object. That is, a character object is attached with a tag indicating that the object is a character, a VFX object is attached with a tag indicating that the object is VFX, and a background object is attached with a tag indicating that the object is background.

[0029] In addition, in this embodiment, a tag definition of which category tag corresponds to which object's mask area is determined in advance, and the tag definition information is stored in memory 104 or the like. That is, the tag definition information has information on the definition that when creating a mask, an object tagged as a character is set as a mask area corresponding to the character (first area specified in green), an object tagged as a VFX is set as a mask area corresponding to VFX (second area specified in blue), and an object tagged as a background is set as a mask area corresponding to the background (third area specified in red). Therefore, for example, when applying an effect to the image 200 shown in FIG. 2, the mask data 300 shown in FIG. 3 is generated, and when the mask data 300 is generated, a mask is generated with each object in the image 200 as a corresponding mask area. That is, in the mask data 300, the region corresponding to the character 201 becomes the mask region, but the tag definition information is referenced based on the tag (tag indicating the character) attached to the character 201, and the region corresponding to the character 201 in the mask data 300 is set as a first region 301 specified in green. Similarly, for the VFX 202, the tag definition information is referenced based on the tag (tag indicating VFX) attached to the VFX 202 in the mask data 300, and the region corresponding to the VFX 202 in the mask data 300 is set as a second region 302 specified in blue, and for the background 203, the tag definition information is referenced based on the tag (tag indicating the background) attached to the background 203 in the mask data 300, and the region corresponding to the background 203 in the mask data 300 is set as a third region 303 specified in red.

[0030] In this embodiment, the first effect applied to the character is contour drawing, which draws the character's outline, the second effect applied to the VFX is color grading (adjusting hue, saturation, brightness, etc.), and the third effect applied to the background is a distortion effect (shifting the output pixels for VFX to create a special effect), but these are not limited to these.

[0031] When processing of a certain frame is started, the process shown in FIG. 4 is executed. Here, image 200 is taken as an example of the "image corresponding to the certain frame." In step S41, the CPU 101 causes the GPU 102 to execute a process of drawing a mask for each object of the image 200 to be currently processed based on the tag definition information. That is, the GPU 102 receives an instruction from the CPU 101 and draws a mask area for the corresponding object in a mask buffer (mask texture) in a color (a color associated with each category of the object to which the effect is applied) designated for each object to which the effect is applied based on the tag definition information, thereby generating mask data 300. The image 200 has a character 201, a VFX 202, and a background 203 arranged as objects, and by referring to the tag definition information, the character 201 is drawn in the mask buffer as a green mask area (first area) 301 since it is a character, the VFX 202 is drawn in the mask buffer as a blue mask area (second area) 302 since it is a VFX, and the background 203 is drawn in the mask buffer as a red mask area (third area) 303 since it is a background.

[0032] In step S42, the CPU 102 causes the GPU 102 to execute a process of drawing each object of the image 200 currently being processed in the camera buffer (a texture to be finally output to the screen) and generating scene data (image data corresponding to the image 200). That is, the CPU 102 draws the corresponding objects, the character 201, the VFX 202, and the background 203, in the camera buffer so as to form an image to be output.

[0033] In step S43, the pixels constituting the scene data are scanned in order to apply a desired effect to each object in the scene data using the mask data 300 generated in step S41 and the scene data generated in step S42. That is, in this embodiment, the pixels of the scene data are scanned one by one to determine which area the corresponding pixel corresponds to, so that the CPU 101 advances the scan of the scene data by one pixel along the scanning direction.

[0034] In step S44a, CPU 101 refers to mask data 300 and determines whether the pixel (target pixel) advanced to in step S43 corresponds to first region 301. More specifically, CPU 101 uses mask data to determine whether the target pixel corresponds to first region 301 based on color information of a pixel corresponding to the position of the target pixel in the scene data. That is, CPU 101 acquires color information of a pixel in mask data 300 corresponding to the position of the target pixel in the scene data, and determines whether the acquired color information is green (the color set in first region 301). If it is green, CPU 101 determines that the target pixel corresponds to first region 301 (the target pixel is included in character object 201), and proceeds to step S45a. If it is not green, CPU 101 determines that the target pixel does not correspond to first region 301, and proceeds to step S44b.

[0035] In step S45a, the CPU 101 applies the first effect to the corresponding pixel determined in step S44a.

[0036] In step S44b, CPU 101, similarly to step S44a, refers to mask data 300 and determines whether or not the pixel in question corresponds to second region 302. That is, CPU 101 determines whether or not color information of a pixel in mask data 300 corresponding to the pixel in question in the scene data is blue, and if it is blue, CPU 101 determines that the pixel in question corresponds to second region 302 and proceeds to step S45b. If it is not blue, CPU 101 determines that the pixel in question does not correspond to second region 302 and proceeds to step S44c.

[0037] In step S45b, CPU 101 applies the second effect to the corresponding pixel determined in step S44b.

[0038] In step S44c, CPU 101 refers to mask data 300 and judges whether or not the pixel corresponds to the third region 303, similarly to steps S44a and S44b. That is, CPU 101 judges whether or not the color information of the pixel in mask data 300 corresponding to the pixel in the scene data is red, and if it is red, CPU 101 judges that the pixel corresponds to the third region 303 and proceeds to step S45c. If it is not red, CPU 101 judges that the pixel does not correspond to the third region 303 and proceeds to step S46. Note that if the pixel is not in the third region 303 in step S44c (if No in this step), the pixel is not in any of the first to third regions, and the effect is not applied to the pixel. Therefore, in this case, the predetermined effect is not applied to the pixel, and the process proceeds to step S46. That is, this pixel is a pixel in the mask data 300 that corresponds to the default color (for example, black) of the mask buffer, and is included in an area of ​​the image to which no effect is applied.

[0039] In step S45c, the CPU 101 applies the third effect to the corresponding pixel determined in step S44c.

[0040] In step S46, CPU 101 determines whether or not the processes in steps 44 and 45 have been performed for all pixels constituting the scene data, and if all pixels have been processed, ends the effect application process for this frame. On the other hand, if all pixels have not been processed, CPU 101 returns to step S43, continues scanning, and processes the next pixel.

[0041] In this way, according to the present embodiment, a mask corresponding to each object to which an effect is to be applied is generated, a mask area is determined for each pixel of an image to be drawn, and an effect corresponding to the determined mask area is applied, so that different effects can be applied according to the category of an object arranged in an image. In addition, for each pixel constituting an image, a corresponding mask area is determined by referring to mask data having a mask area corresponding to each object, so that even if there are many object categories, a corresponding mask area can be determined for each of all pixels by scanning the entire screen once. In other words, even if the number of object categories increases, different effects according to the object categories can be applied by scanning the screen once, so that an arbitrary effect can be applied to an arbitrary object with reduced processing load.

[0042] Furthermore, in this embodiment, since the colors for specifying the mask area of ​​the mask data are RGB, area determination can be simplified.

[0043] In this embodiment, the number of object categories is three (character, VFX, background), but if there are four or more categories, the colors specified in the mask areas corresponding to each object category can be different from each other.

[0044] In addition, in this embodiment, it is no exaggeration to say that the effect to be applied is determined by the color specified in the mask used to apply the effect. That is, the color of the mask area determines which mask area each pixel constituting the image corresponds to, and the effect to be applied is determined accordingly. In the above example, such colors are determined for each object category, but when generating the mask area, different colors may be specified for different objects in the same category. That is, in this embodiment, different colors can be assigned to the first character and the second character, and by doing so, different effects can be applied to different objects in the same category.

[0045] Furthermore, in this embodiment, it is not necessary to apply a predetermined effect to all objects appearing in a certain image (that is, there may be objects to which no effect is applied).

[0046] (Other embodiments) The above-mentioned embodiment also includes a processing method in which a program (for example, a program that performs the processing shown in FIG. 4) that operates the configuration of the above-mentioned embodiment to realize the functions of the above-mentioned embodiment is stored in a storage medium, the program stored in the storage medium is read as code, and the program is executed by a computer. That is, a computer-readable storage medium is also included in the scope of the embodiment. In addition, the above-mentioned embodiment includes not only the storage medium in which the above-mentioned computer program is stored, but also the computer program itself. Examples of such a storage medium include a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a non-volatile memory card, a ROM, and the like. In addition, the above-mentioned embodiment also includes not only a program stored in the above-mentioned storage medium that executes processing by itself, but also a program that operates on an OS in cooperation with other software and functions of an extension board to execute the operations of the above-mentioned embodiment. [Explanation of symbols]

[0047] 100 Information processing device 101 CPU 102 GPU 103 Storage 104 Memory 200 images 201 Characters 202 VFX 203 Background 300 Mask Data 301 First Area 302 Second Area 303 The Third Region

Claims

1. An information processing device capable of applying an effect to at least one object included in a predetermined image generated during a game, means for drawing a mask area corresponding to each of the objects to which the effect is applied in a mask buffer, the mask area being drawn in a color associated with each category of the object to which the effect is applied, to generate mask data; means for drawing the corresponding object in a camera buffer so as to form an image to be output; a means for scanning pixels of an entire screen of the predetermined image drawn in the camera buffer using the mask data as a mask, each of the colors being associated with an effect to be applied, and for the scanned pixels, applying a corresponding effect to a pixel to which the effect should be applied, by determining a color of a mask area for each of the scanned pixels; If the color is not associated with the effect to be applied as a result of the determination, the effect is not applied to the corresponding pixel; means for applying an effect corresponding to the determined color of each pixel when the determined color is associated with the effect to be applied; An image processing device comprising:

2. An information processing method capable of applying an effect to at least one object included in a predetermined image generated during a game, comprising: a step of drawing a mask area corresponding to each of the objects to which the effect is applied in a mask buffer, the mask area being drawn in a color associated with each category of the object to which the effect is applied, thereby generating mask data; Rendering the corresponding object into a camera buffer to form an image to be output; a means for scanning pixels of an entire screen of the predetermined image drawn in the camera buffer using the mask data as a mask, each of the colors being associated with an effect to be applied, and for the scanned pixels, applying a corresponding effect to a pixel to which the effect should be applied, by determining a color of a mask area for each of the scanned pixels; If the color is not associated with the effect to be applied as a result of the determination, the effect is not applied to the corresponding pixel; applying an effect corresponding to the determined color for each pixel when the determined color is associated with the effect to be applied; 13. An image processing method comprising:

3. A program that causes a computer to function as an information processing device that is capable of applying an effect to at least one object included in a predetermined image when generating the image in a game, the program comprising: The information processing device includes: means for drawing a mask area corresponding to each of the objects to which the effect is applied in a mask buffer, the mask area being drawn in a color associated with each category of the object to which the effect is applied, to generate mask data; means for drawing the corresponding object in a camera buffer so as to form an image to be output; a means for scanning pixels of an entire screen of the predetermined image drawn in the camera buffer using the mask data as a mask, each of the colors being associated with an effect to be applied, and for the scanned pixels, applying a corresponding effect to a pixel to which the effect should be applied, by determining a color of a mask area for each of the scanned pixels; If the color is not associated with the effect to be applied as a result of the determination, the effect is not applied to the corresponding pixel; means for applying an effect corresponding to the determined color of each pixel when the determined color is associated with the effect to be applied; A program comprising:

4. A computer-readable storage medium storing the program according to claim 3.

Images