Image generation device, image generation method, and program

Abstract

An image generation device, an image generation method, and a program are provided, which can save the storage capacity of a frame buffer for storing pixel values of pixels whose pixel values are predetermined to be a specified value. An acquisition management data storage unit (30) stores acquisition management data. A pixel value storage unit (32) stores pixel values of pixels whose pixel values are determined to be the pixels acquired from the pixel value storage unit (32). An acquisition determination unit (34) determines whether to acquire the pixel value of each pixel from the pixel value storage unit (32) based on the acquisition management data. For pixels determined to acquire their pixel values from the pixel value storage unit (32), a pixel value determination unit (38) determines to acquire the pixel values from the pixel value storage unit (32) as the pixel values of these pixels. For pixels determined not to acquire their pixel values from the pixel value storage unit (32), the pixel value determination unit (38) determines the specified value as the pixel values of these pixels.

Description

Technical Field

[0001] The present invention relates to an image generation device, an image generation method, and a program. Background Art

[0002] In some cases, an overlay image to be drawn in a frame buffer is generated at a predetermined frame rate, and in each overlay image, a background image representing a situation in a playing game and a foreground image (such as characters, shapes, or symbols) representing a user interface in the game are superimposed on each other.

[0003] In addition, in some cases, in order to achieve stereoscopic vision through a head-mounted display including an optical system, a rectangular three-dimensional image to be drawn in a frame buffer is generated, which includes a distortion-corrected left-eye image and a distortion-corrected right-eye image in a circular field-of-view range of a user's left eye and a circular field-of-view range of the user's right eye, respectively. Summary of the Invention

[0004] [Technical Problem]

[0005] For example, in some cases, in the above foreground image, an area where a two-dimensional object is to be arranged is determined in advance by a designer or the like, and a pixel value representing no color and full transparency is set in advance as a pixel value for pixels outside the area.

[0006] In addition, for example, in the above rectangular three-dimensional image, a pixel value representing black is set as a pixel value for pixels outside the circular field-of-view range.

[0007] Here, for example, for a pixel whose pixel value is predetermined as a predetermined value (such as a pixel value representing no color and full transparency or a pixel value representing black), it is not necessary to ensure a storage capacity in the frame buffer as large as that of other pixels. For example, regarding an image of pixels that require 32 bits of storage capacity per pixel to store their color and transparency, for a pixel whose pixel value is predetermined as a predetermined value, it is not necessary to ensure 32 bits of storage capacity in the frame buffer.

[0008] In view of the above circumstances, the present invention has been made, and an object of the present invention is to provide an image generation device, an image generation method, and a program that can save the storage capacity of a frame buffer for storing pixel values of pixels whose pixel values are predetermined as a predetermined value.

[0009] [Solution to the Problem]

[0010] In order to solve the above problems, according to the present invention, there is provided an image generation device, including: a pixel value storage unit configured to store pixel values; an acquisition management data storage unit configured to store acquisition management data, the acquisition management data indicating whether to acquire the pixel values of each pixel included in the image from the pixel value storage unit; an acquisition determination unit configured to determine whether to acquire the pixel values of each pixel from the pixel value storage unit based on the acquisition management data; a pixel value acquisition unit configured to acquire the pixel values of pixels determined to have pixel values acquired from the pixel value storage unit from the pixel value storage unit; and a pixel value determination unit configured to determine the pixel values of each pixel. The pixel value storage unit stores the pixel values of pixels determined to have pixel values acquired from the pixel value storage unit. The pixel value determination unit determines the pixel value acquired from the pixel value storage unit as the pixel value of a pixel determined to have a pixel value acquired from the pixel value storage unit. The pixel value determination unit determines a predetermined value as the pixel value of a pixel determined to have a pixel value not acquired from the pixel value storage unit.

[0011] In one aspect of the present invention, the acquisition management data includes a plurality of individual acquisition management data, each piece of data being associated with a plurality of pixels. The acquisition determination unit determines whether to acquire the pixel values of the plurality of pixels associated with the individual acquisition management data from the pixel value storage unit based on the individual acquisition management data. The pixel value determination unit determines a predetermined value as the pixel values of all pixels among the plurality of pixels associated with the individual acquisition management data and determined to have pixel values not acquired from the pixel value storage unit.

[0012] In this aspect, the individual acquisition management data may be associated with a plurality of pixels occupying a rectangular area in the image.

[0013] Furthermore, in one aspect of the present invention, the pixel value storage unit stores pixel values and alpha values. The pixel value determination unit determines the pixel value and alpha value acquired from the pixel value storage unit as the pixel value and alpha value of a pixel determined to have a pixel value acquired from the pixel value storage unit. The pixel value determination unit determines a predetermined value as the pixel value and the alpha value of a pixel determined to have a pixel value not acquired from the pixel value storage unit. The image generation device further includes an image generation unit configured to generate a superimposed image by superimposing an image having pixel values and alpha values determined by the pixel value determination unit and another image on each other.

[0014] In addition, in one aspect of the present invention, the acquisition management data indicates not to acquire the pixel values of pixels occupying a given area in the image from the pixel value storage unit.

[0015] In addition, according to the present invention, there is provided an image generation method, including: a pixel value storage step of storing pixel values in a pixel value storage unit; an acquisition management data storage step of storing acquisition management data in an acquisition management data storage unit, the acquisition management data indicating whether to acquire the pixel value of each pixel included in the image from the pixel value storage unit; an acquisition determination step of determining whether to acquire the pixel value of each pixel from the pixel value storage unit based on the acquisition management data; a pixel value acquisition step of acquiring the pixel value of a pixel determined to have a pixel value acquired from the pixel value storage unit from the pixel value storage unit; and a pixel value determination step of determining the pixel value of each pixel. In the pixel value storage step, the pixel value of a pixel determined to have a pixel value acquired from the pixel value storage unit is stored in the pixel value storage unit. In the pixel value determination step, for a pixel determined to have a pixel value acquired from the pixel value storage unit, the pixel value acquired from the pixel value storage unit is determined as the pixel value of the pixel. In the pixel value determination step, for a pixel determined to have a pixel value not acquired from the pixel value storage unit, a predetermined value is determined as the pixel value of the pixel.

[0016] In addition, according to the present invention, there is provided a program for causing a computer to execute: a pixel value storage process of storing pixel values in a pixel value storage unit; an acquisition management data storage process of storing acquisition management data indicating whether to acquire the pixel value of each pixel included in the image from the pixel value storage unit in an acquisition management data storage unit; an acquisition determination process of determining whether to acquire the pixel value of each pixel from the pixel value storage unit based on the acquisition management data; a pixel value acquisition process of acquiring the pixel value of a pixel determined to have a pixel value acquired from the pixel value storage unit from the pixel value storage unit; and a pixel value determination process of determining the pixel value of each pixel. In the pixel value storage process, the pixel value of a pixel determined to have a pixel value acquired from the pixel value storage unit is stored in the pixel value storage unit. In the pixel value determination process, for a pixel determined to have a pixel value acquired from the pixel value storage unit, the pixel value acquired from the pixel value storage unit is determined as the pixel value of the pixel. In the pixel value determination process, for a pixel determined to have a pixel value not acquired from the pixel value storage unit, a predetermined value is determined as the pixel value of the pixel. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a diagram showing an example of the configuration of an image processing apparatus according to an embodiment of the present invention.

[0018] Figure 2 is a diagram schematically showing an example of the data structure of acquisition management data associated with a foreground image.

[0019] Figure 3 is a diagram schematically showing an example of a data structure for acquiring management data associated with a three-dimensional image.

[0020] Figure 4 is a functional block diagram showing an example of the functions of an image processing apparatus according to an embodiment of the present invention.

[0021] Figure 5 is a flowchart showing an example of the flow of processing executed by an image processing apparatus according to an embodiment of the present invention.

[0022] Figure 6 is a flowchart showing an example of the flow of processing executed by an image processing apparatus according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] Now, an embodiment of the present invention will be described with reference to the accompanying drawings.

[0024] Figure 1 is a diagram showing an example of the configuration of an image processing apparatus 10 according to an embodiment of the present invention. The image processing apparatus 10 according to the present embodiment is, for example, a game console, a handheld game console, or a personal computer. As Figure 1 shown, the image processing apparatus 10 according to the present embodiment includes a processor 12, a storage unit 14, an operation unit 16, and a display unit 18.

[0025] The processor 12 is a program control device such as a CPU (Central Processing Unit), and operates according to a program installed in the image processing apparatus 10. The processor 12 according to the present embodiment further includes a GPU (Graphics Processing Unit), which is configured to draw an image in a frame buffer based on a graphics command or data provided from the CPU.

[0026] The storage unit 14 is a storage element such as a ROM (Read Only Memory) or a RAM (Random Access Memory), a hard disk drive, etc. Further, the storage unit 14 according to the present embodiment has a frame buffer area in which an image is drawn by the GPU.

[0027] The operation unit 16 is a keyboard, a mouse, a game console controller, etc. The operation unit 16 receives an operation input by a user and outputs a signal indicating the content of the operation to the processor 12.

[0028] The display unit 18 is a display device such as a liquid crystal display, and displays various images according to an instruction from the processor 12. Further, the display unit 18 may be an HMD (Head Mounted Display).

[0029] Now, a description will be given according to the use of Figure 2 and Figure 3This embodiment of obtaining the management data 20 illustrates an example of how to save the storage capacity of the frame buffer in the image processing apparatus 10 for storing pixel values included in an image.

[0030] Figure 2 FIG. is a diagram schematically showing an example of obtaining the management data 20 according to this embodiment.

[0031] Here, for example, it is assumed that superimposed images in which a background image and a foreground image are superimposed on each other are generated at a predetermined frame rate (for example, every 1 / 60 second). Here, the background image may be, for example, a frame image representing a situation in a playing game obtained when a virtual three-dimensional object representing a game character in a virtual three-dimensional space is viewed from a viewpoint in the virtual three-dimensional space. In addition, the foreground image may be, for example, a frame image representing information about a user interface in the game. Here, the foreground image may be a frame image representing a two-dimensional object such as a character, a shape, or a symbol. In addition, Figure 2 The obtained management data 20 shown in FIG. is an example of the management data 20 associated with the foreground image.

[0032] For example, since the situation obtained when a virtual three-dimensional object in a virtual three-dimensional space is viewed from a viewpoint in a playing game is often expressed graphically, it is likely to allow a blurred expression to some extent. At the same time, when the resolution of a frame image in which two-dimensional objects such as characters, shapes, or symbols are arranged is low, the frame image is blurred, and as a result, it is difficult for the user to read the content.

[0033] In view of this, in the following description, for example, it is assumed that the resolution of the foreground image is higher than the resolution of the background image. In addition, for example, it is assumed that the superimposed image is generated by adjusting the size of the background image to match the resolution of the foreground image and then superimposing the foreground image and the resized background image on each other. In the following description, for example, it is assumed that the superimposed image is generated by performing alpha blending on the foreground image and the resized background image.

[0034] In addition, in the following description, for example, in the storage unit 14, a frame buffer area for storing pixel values of the background image and a frame buffer area for storing pixel values and alpha values of the foreground image are provided separately. The frame buffer storing the pixel values of the background image is hereinafter referred to as the "background frame buffer", and the frame buffer storing the pixel values and alpha values of the foreground image is hereinafter referred to as the "foreground frame buffer".

[0035] Here, for example, assume that a pixel stored in the background frame buffer has a pixel value size of 24 bits. Additionally, for example, assume that a pixel stored in the foreground frame buffer has a pixel value size of 24 bits and an alpha value size of 8 bits. Therefore, the storage capacity of the foreground frame buffer for storing the pixel value and alpha value of a pixel is 32 bits. Note that the pixel value sizes of the red, green, and blue components of a pixel can each be 8 bits.

[0036] Figure 2 The acquisition management data 20 shown in includes a plurality of individual acquisition management data 22. Figure 2 The individual acquisition management data 22 shown in Figure 2 is associated with a pixel block obtained when the foreground image is divided into a plurality of pixel blocks, where each pixel block has a plurality of pixels belonging to it. Here, for example, assume that the number of pixels belonging to a pixel block is 14,000, and the total pixel value size of the pixels belonging to the pixel block is 56 kilobytes.

[0037] Additionally, for example, a pixel block can be a plurality of pixels occupying a rectangular area in the foreground image. Additionally, each position of the individual acquisition management data 22 included in the acquisition management data 20 Figure 2 shown in Figure 2 is associated with the position of the pixel block associated with the individual acquisition management data 22 in the foreground image.

[0038] Figure 2 The acquisition management data 20 shown in Figure 2 is, for example, data indicating whether to acquire the pixel value and alpha value from the foreground frame buffer. Here, for example, one piece of individual acquisition management data 22 included in the acquisition management data 20 can be, for example, a 1-bit flag indicating whether to acquire the pixel value and alpha value from the foreground frame buffer. Additionally, for example, the individual acquisition management data 22 indicating that the pixel value and alpha value are to be acquired from the foreground frame buffer can have a value of 1. Additionally, the individual acquisition management data 22 indicating that the pixel value and alpha value are not to be acquired from the foreground frame buffer can have a value of 0.

[0039] Hereinafter, the individual acquisition management data 22 with a value of 1 will be referred to as "first-type individual acquisition management data 22a", and the individual acquisition management data 22 with a value of 0 will be referred to as "second-type individual acquisition management data 22b". In Figure 2 the example of Figure 2 , the hatched individual acquisition management data 22 corresponds to the first-type individual acquisition management data 22a, and the non-hatched individual acquisition management data 22 corresponds to the second-type individual acquisition management data 22b.

[0040] In Figure 2In the example, for instance, it is assumed that a designer or the like has preset an area in the foreground image where information about the user interface can be placed, and a predetermined value such as a pixel value representing no color and full transparency has been set for pixels outside this area. In this case, for example, as Figure 2 shown, the first type of individual acquisition management data 22a is set as the individual acquisition management data 22 associated with an area in the foreground image where information such as the user interface can be placed. In Figure 2 the example, the first type of individual acquisition management data 22a is set as the individual acquisition management data 22 associated with the lower left rectangular area. In addition, the second type of individual acquisition management data 22b is set as the individual acquisition management data 22 associated with pixels outside the rectangular area.

[0041] Here, in Figure 2 the example, for example, regarding the pixels associated with the first type of individual acquisition management data 22a, the foreground frame buffer storing the pixel values and alpha values of the pixels is accessed. Here, for example, for 14,000 pixels associated with one piece of the first type of individual acquisition management data 22a, the foreground frame buffer storing the pixel values and alpha values of each of the 14,000 pixels is accessed. Then, the pixel values and alpha values of each of the 14,000 pixels are acquired. Thus, for the 14,000 pixels, the acquired pixel values and alpha values are determined as the pixel values and alpha values of the pixels.

[0042] Meanwhile, for the pixels associated with the second type of individual acquisition management data 22b, the foreground frame buffer is not accessed. In addition, for all 14,000 pixels associated with the second type of individual acquisition management data 22b, the predetermined value is determined as the pixel values and alpha values of the pixels. Here, for example, the pixel values and alpha values representing no color and full transparency can be determined as the pixel values and alpha values of the pixels. Thus, in the pixels associated with the second type of individual acquisition management data 22b, the content of the background image appears in the superimposed image.

[0043] Figure 3 is a diagram schematically showing another example of the acquisition management data 20 according to the present embodiment.

[0044] Here, for example, it is assumed that a three-dimensional image is generated at a predetermined frame rate (e.g., every 1 / 60 second). The three-dimensional image is, for example, a rectangular image, and in order to achieve a stereoscopic image through a head-mounted display including an optical system, the rectangular image includes a distortion-corrected left-eye image and a distortion-corrected right-eye image within the circular field of view of the user's left eye and the circular field of view of the user's right eye, respectively. In addition, Figure 3 the acquisition management data 20 shown in

[0045] Figure 3 The acquisition management data 20 shown therein includes a plurality of individual acquisition management data 22. Figure 3 The individual acquisition management data 22 shown is associated with pixel blocks obtained when a three-dimensional image is divided into a plurality of pixel blocks, where each pixel block has a plurality of pixels belonging to it.

[0046] In addition, for example, a pixel block can be a plurality of pixels occupying a rectangular area in the foreground image. In addition, included in the acquisition management data 20 Figure 3 The position of each of the individual acquisition management data 22 shown is associated with the position of the pixel block associated with the individual acquisition management data 22 in the foreground image.

[0047] Figure 3 The acquisition management data 20 shown therein is, for example, data indicating whether to acquire pixel values from a frame buffer. Here, for example, one piece of individual acquisition management data 22 included in the acquisition management data 20 can be, for example, a 1-bit flag indicating whether to acquire pixel values from a frame buffer. In addition, for example, the individual acquisition management data 22 indicating that pixel values are to be acquired from the frame buffer can have a value of 1, and the individual acquisition management data 22 indicating that pixel values are not to be acquired from the frame buffer can have a value of 0.

[0048] Similarly in Figure 3 In the example of, the individual acquisition management data 22 with a value of 1 is referred to as "first type of individual acquisition management data 22a", and the individual acquisition management data 22 with a value of 0 is referred to as "second type of individual acquisition management data 22b". Similarly in Figure 3 In the example of, the hatched individual acquisition management data 22 corresponds to the first type of individual acquisition management data 22a, and the non-hatched individual acquisition management data 22 corresponds to the second type of individual acquisition management data 22b.

[0049] In addition, Figure 3 Two circular viewing areas 24 associated with the circular viewing field range of the user's left eye and the circular viewing field range of the user's right eye in the three-dimensional image are shown. In this embodiment, for example, it is assumed that the viewing area 24 is associated with a given area in the three-dimensional image. In addition, in Figure 3 In the example of, the second type of individual acquisition management data 22b is set as the individual acquisition management data 22 associated with pixel blocks that only include pixels outside the viewing area 24. In addition, the first type of individual acquisition management data 22a is set as the individual acquisition management data 22 associated with pixel blocks that include some or all of the pixels in the viewing area 24.

[0050] In Figure 3In the example, for the pixels associated with the first type of individual acquisition management data 22a, the frame buffer storing the pixel values of the pixels is accessed. Here, for example, for a plurality of pixels associated with one piece of the first type of individual acquisition management data 22a, the frame buffer storing the pixel values of each of the plurality of pixels is accessed. Then, for each of the plurality of pixels, the pixel value of the pixel is acquired. Then, for the plurality of pixels, the acquired pixel values are determined as the pixel values of the pixels.

[0051] At the same time, for the pixels associated with the second type of individual acquisition management data 22b, the frame buffer storing the pixel values of the pixels is not accessed. For the pixel, a predetermined value is determined as the pixel value of the pixel. Here, for example, a pixel value representing black can be determined as the pixel value of the pixel. Thus, in the three-dimensional image, the area outside the viewing area 24 is black.

[0052] In the three-dimensional image, a predetermined value such as a pixel value representing black is preset as the pixel value of the pixels in the area outside the circular field of view range. Therefore, for the pixels associated with the second type of individual acquisition management data 22b associated with a pixel block including only the pixels outside the viewing area 24 as described above, a pixel value representing black is determined as the pixel value of the pixel without any problem.

[0053] In the present embodiment, in Figure 2 the acquisition management data 20 shown and Figure 3 the acquisition management data 20 shown, the storage capacity of the frame buffer for storing the pixel values of the pixels associated with the first type of individual acquisition management data 22a is ensured. At the same time, the storage capacity of the frame buffer for storing the pixel values of the pixels associated with the second type of individual acquisition management data 22b is not ensured. In addition, in the present embodiment, for example, the area in the storage unit 14 corresponding to the storage capacity of the frame buffer for storing the pixel values of the pixels associated with the second type of individual acquisition management data 22b can be used for purposes other than the frame buffer.

[0054] Note that the acquisition management data 20 itself can be stored in the area in the storage unit 14 ensured for the frame buffer, or can be stored outside the area in the storage unit 14 ensured for the frame buffer.

[0055] As described above, according to the present embodiment, the storage capacity of the frame buffer for storing the pixel values of the pixels predetermined to have a predetermined value as their pixel values can be saved. For example, in the example described with reference to Figure 2 compared with the case where the pixel values and alpha values of all the pixels included in the foreground image are stored in the foreground frame buffer, the size of the area in the storage unit 14 ensured for the foreground frame buffer can be saved. In addition, for example, in the example described with reference to Figure 3In the described example, compared with the case where the pixel values of all pixels included in the three-dimensional image are stored in the frame buffer, the size of the area in the storage unit 14 that is ensured for the frame buffer can be saved.

[0056] Note that although the storage unit 14 is required for storing the acquisition management data 20, the storage capacity of the storage unit 14 required for storing the acquisition management data 20 is smaller than the storage capacity of the storage unit 14 required for storing the pixel values. Therefore, even if the acquisition management data 20 is stored in the area in the storage unit 14 that is ensured for the frame buffer, the storage capacity of the frame buffer can be saved as a whole.

[0057] In addition, for example, in the example described in the reference Figure 2 only the pixel values can be stored in the foreground frame buffer, and the alpha values may not be stored. In addition, in this case, for example, the superimposed image can be generated by overlaying the foreground image on the resized background image.

[0058] Now, the functions of the image processing apparatus 10 according to the present embodiment and the processing executed by the image processing apparatus 10 according to the present embodiment will be further described, with emphasis on the determination of the pixel values using the acquisition management data 20.

[0059] Figure 4 is a functional block diagram showing an example of the functions implemented on the image processing apparatus 10 according to the present embodiment. Note that on the image processing apparatus 10 according to the present embodiment, not necessarily all the functions shown Figure 4 are implemented, and functions other than the functions shown Figure 4 may be implemented.

[0060] As shown Figure 4 in, the image processing apparatus 10 functionally includes, for example, an acquisition management data storage unit 30, a pixel value storage unit 32, an acquisition determination unit 34, a pixel value acquisition unit 36, a pixel value determination unit 38, an image generation unit 40, and a display control unit 42. The acquisition management data storage unit 30 and the pixel value storage unit 32 are mainly implemented by the storage unit 14. The acquisition determination unit 34, the pixel value acquisition unit 36, the pixel value determination unit 38, and the image generation unit 40 are mainly implemented by the processor 12, and the display control unit 42 is mainly implemented by the processor 12 and the display unit 18. The image processing apparatus 10 functions as an image generation apparatus configured to generate an image in the present embodiment.

[0061] The above functions can be implemented by the processor 12 executing a program installed in the image processing apparatus 10 which is a computer and includes instructions corresponding to the above functions. The program can be provided to the image processing apparatus 10 through a computer-readable information storage medium such as an optical disk, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory, or through the Internet or the like.

[0062] The acquisition management data storage unit 30 stores the acquisition management data 20, which indicates whether to acquire the pixel value of each pixel included in the image in the present embodiment, for example, from the pixel value storage unit 32.

[0063] Here, the acquisition management data 20 may include a plurality of individual acquisition management data 22, each associated with a plurality of pixels. In addition, as referred to Figure 2 above, the individual acquisition management data 22 may be associated with a plurality of pixels that occupy a rectangular area in the image. In Figure 2 and Figure 3 the example of, the individual acquisition management data 22 is associated with a plurality of pixels (pixel blocks) in a rectangular area in the image. Note that one piece of individual acquisition management data 22 may be associated with one pixel.

[0064] In addition, the acquisition management data 20 may indicate not to acquire the pixel value of the pixels that occupy a given area in the image from the pixel value storage unit 32. For example, in Figure 2 the example of, the individual acquisition management data 22 associated with the pixels that occupy a given area outside the area where a two-dimensional object can be arranged may indicate not to access the pixel value storage unit 32. In addition, for example, in Figure 3 the example of, the individual acquisition management data 22 associated with the pixels that occupy a given area outside a given circular field of view range associated with the viewing area 24 may indicate not to access the pixel value storage unit 32.

[0065] For example, in the present embodiment, the pixel value storage unit 32 stores pixel values. Here, for example, the pixel value storage unit 32 may store the pixel values of the pixels determined by the acquisition determination unit 34 to have pixel values acquired from the pixel value storage unit 32.

[0066] In addition, the pixel value storage unit 32 may store the pixel values and alpha values of the pixels determined to have pixel values acquired from the pixel value storage unit 32.

[0067] In addition, in the present embodiment, for example, in the pixel value storage unit 32, the storage capacity of the frame buffer for storing the pixel values of the pixels determined to have pixel values not acquired from the pixel value storage unit 32 is not ensured.

[0068] For example, ensure as for storing reference Figure 2The foreground frame buffer of the pixel values and alpha values in the described foreground image or the area of the frame buffer for storing the pixel values in the described three-dimensional image, corresponding to the pixel value storage unit 32. Figure 3 The area of the frame buffer for storing the pixel values in the described three-dimensional image, corresponding to the pixel value storage unit 32.

[0069] The acquisition determination unit 34 determines whether to acquire the pixel value of each pixel included in the image from, for example, the pixel value storage unit 32 based on the acquired management data 20. Here, the acquisition determination unit 34 can also determine whether to acquire the pixel values of a plurality of pixels corresponding to the individual acquisition management data 22 from the pixel value storage unit 32 according to the individual acquisition management data 22. For example, it can be determined to acquire the pixel values of the pixels associated with the first type of individual acquisition management data 22a from the pixel value storage unit 32. In addition, it can be determined not to acquire the pixel values of the pixels associated with the second type of individual acquisition management data 22b from the pixel value storage unit 32.

[0070] The pixel value acquisition unit 36 acquires the pixel value of, for example, the pixel determined to have the pixel value acquired from the pixel value storage unit 32 in the present embodiment from the pixel value storage unit 32.

[0071] For example, in the present embodiment, the pixel value determination unit 38 determines the pixel value of each pixel included in the image. Here, for example, the pixel value determination unit 38 determines the pixel value acquired from the pixel value storage unit 32 as the pixel value of the pixel determined to have the pixel value acquired from the pixel value storage unit 32. In addition, for example, the pixel value determination unit 38 determines a predetermined value as the pixel value of the pixel determined to have a pixel value not acquired from the pixel value storage unit 32.

[0072] Here, the pixel value determination unit 38 can determine the pixel value and the alpha value. For example, for the pixel determined to have the pixel value acquired from the pixel value storage unit 32, the pixel value and the alpha value acquired from the pixel value storage unit 32 can be determined as the pixel value and the alpha value of the pixel. In addition, for example, for the pixel determined to have a pixel value not acquired from the pixel value storage unit 32, a predetermined value can be determined as the pixel value and the alpha value of the pixel. In the reference Figure 2 In the described example, the predetermined value is the pixel value and the alpha value representing colorless and fully transparent. In addition, in the example referred to Figure 3 In the described example, the predetermined value is the pixel value representing black.

[0073] In addition, the pixel value determination unit 38 may determine a predetermined value as the pixel values of all the plurality of pixels associated with the second type of individual acquisition management data 22b that are determined to have pixel values not acquired from the pixel value storage unit 32. Here, for all the plurality of pixels associated with the second type of individual acquisition management data 22b, the predetermined value may be determined as the pixel values and alpha values of the plurality of pixels.

[0074] The image generation unit 40 generates an image based on, for example, the pixel values determined by the pixel value determination unit 38 in the present embodiment. Here, the image generation unit 40 may generate a superimposed image by superimposing an image having the pixel values and alpha values determined by the pixel value determination unit 38 and another image on each other. For example, as described with reference to Figure 2 the image generation unit 40 may generate a superimposed image by superimposing a foreground image having the pixel values determined by the pixel value determination unit 38 and a background image having pixels with pixel values stored in the background frame buffer on each other. In addition, a superimposed image may be generated by superimposing the foreground image and a resized background image on each other.

[0075] In addition, for example, the pixel value determination unit 38 may determine the pixel value of each pixel included in the superimposed image. Here, for example, alpha blending based on the alpha value of the pixel in the foreground image may be performed. For example, based on the alpha value of the pixel included in the foreground image, the pixel value of the pixel, and the pixel value of the pixel associated with the pixel in question in the background image, the pixel value of the pixel associated with the pixel in question in the superimposed image may be determined. Then, the image generation unit 40 may generate a superimposed image based on the pixel values thus determined.

[0076] In addition, as described with reference to Figure 3 the image generation unit 40 may generate a three-dimensional image.

[0077] The display control unit 42 controls the display unit 18 to display, for example, the image generated by the image generation unit 40 of the present embodiment. Here, for example, the superimposed image described with reference to Figure 2 may be displayed on the display. In addition, for example, the three-dimensional image described with reference to Figure 3 may be displayed on the HMD.

[0078] Here, an example of the processing flow executed by the image processing apparatus 10 according to the present embodiment is described with reference to the flowchart of Figure 5 The processing described in this processing example corresponds to the example described with reference to Figure 2 the description.

[0079] Note that, in this processing example, it is assumed that the acquisition management data 20 in the frame is stored in the acquisition management data storage unit 30 at a predetermined frame rate. In addition, in this processing example, it is assumed that the pixel values of the background image in the frame are stored in the background frame buffer at a predetermined frame rate. Further, it is assumed that the pixel values and alpha values in the foreground image associated with the first type of individual acquisition management data 22a in the frame are stored in the foreground frame buffer at a predetermined frame rate.

[0080] First, the acquisition determination unit 34 acquires the acquisition management data 20 in the frame from the acquisition management data storage unit 30 (S101).

[0081] Then, the acquisition determination unit 34 selects, from the individual acquisition management data 22 included in the acquisition management data 20 acquired in the process of S101, the individual acquisition management data 22 that has not undergone the processing of S103 to S107 in the frame (S102).

[0082] Then, the acquisition determination unit 34 confirms the value set for the individual acquisition management data 22 selected in the process of S102 (S103).

[0083] Here, it is assumed that the value is confirmed to be 1, that is, it is confirmed that the individual acquisition management data 22 is the first type of individual acquisition management data 22a. In this case, the pixel value acquisition unit 36 accesses the pixel value storage unit 32 to acquire the pixel value and alpha value of each of the multiple pixels belonging to the pixel block associated with the first type of individual acquisition management data 22a (S104).

[0084] Then, for each of the multiple pixels belonging to the pixel block, the pixel value determination unit 38 determines the pixel value and alpha value of the pixel acquired in the process of S104 as the pixel value and alpha value of the pixel included in the foreground image (S105).

[0085] Meanwhile, it is assumed that the value is confirmed to be 0 in the process of S103. That is, it is confirmed that the individual acquisition management data 22 is the second type of individual acquisition management data 22b. In this case, the pixel value determination unit 38 determines a predetermined value as the pixel value and alpha value of the pixels included in the foreground image for all the pixels belonging to the pixel block associated with the second type of individual acquisition management data 22b (S106). Here, for example, as the pixel value and alpha value of the pixel, the pixel value and alpha value representing no color and full transparency are determined.

[0086] Then, the pixel value determination unit 38 determines the pixel values of the plurality of pixels in the superimposed image associated with the plurality of pixels belonging to the pixel block (S107). Here, for example, the pixel values of the pixels in the superimposed image can be determined based on the pixel values and alpha values of the pixels included in the foreground image determined in the process of S105 or S106 and the pixel values of the pixels in the background image associated with the pixel under discussion.

[0087] Then, the pixel value determination unit 38 confirms whether all the individual acquisition management data 22 in the frame have undergone the processes of S103 to S107 (S108).

[0088] In the case where not all the individual acquisition management data 22 in the frame have undergone the processes of S103 to S107 (S108: No), the process returns to the process of S102.

[0089] In the case where all the individual acquisition management data 22 in the frame have undergone the processes of S103 to S107 (S108: Yes), the image generation unit 40 generates the superimposed image in the frame (S109). Here, for example, the superimposed image is generated based on the pixel values determined in the process of S107.

[0090] Then, the display control unit 42 controls the display unit 18 to display the superimposed image generated in the process of S109 (S110), and the process returns to the process of S101.

[0091] In this processing example, the processes of S101 to S110 are repeatedly executed at a predetermined frame rate.

[0092] Note that, in the above processing example, the acquisition management data 20 in the frame may not be stored in the acquisition management data storage unit 30 at a predetermined frame rate. For example, the acquisition management data 20 common to all frames can be used. In this case, when the process of S110 ends, the process can return to the process of S102, and the process of the next frame can be executed.

[0093] Next, with reference to Figure 6 the flowchart, another example of the processing flow of the image processing apparatus 10 of the present embodiment will be described. The processing described in this processing example corresponds to the example described with reference to Figure 3 described.

[0094] Note that, in this processing example, the acquisition management data 20 in the frame is stored in the acquisition management data storage unit 30 at a predetermined frame rate. In addition, in this processing example, the pixel values of the pixels of the three-dimensional image corresponding to the first type of individual acquisition management data 22a in the frame are stored in the frame buffer at a predetermined frame rate.

[0095] First, acquisition determination unit 34 acquires acquisition management data 20 in the frame from acquisition management data storage unit 30 (S201).

[0096] Then, acquisition determination unit 34 selects, from individual acquisition management data 22 included in acquisition management data 20 acquired in the process of S201, individual acquisition management data 22 that has not undergone the processes of S203 to S206 in this frame (S202).

[0097] Then, acquisition determination unit 34 confirms the value set for individual acquisition management data 22 selected in the process of S202 (S203).

[0098] Here, it is assumed that this value is confirmed to be 1. That is, it is confirmed that individual acquisition management data 22 is first - type individual acquisition management data 22a. In this case, pixel value acquisition unit 36 accesses pixel value storage unit 32 to acquire the pixel value of each of the multiple pixels belonging to the pixel block associated with first - type individual acquisition management data 22a (S204).

[0099] Then, pixel value determination unit 38 determines, for each of the multiple pixels belonging to the pixel block, the pixel value of the pixel acquired in the process of S204 as the pixel value of the pixel included in the three - dimensional image (S205).

[0100] Meanwhile, it is assumed that in the process of S203, this value is confirmed to be 0. That is, it is confirmed that individual acquisition management data 22 is second - type individual acquisition management data 22b. In this case, pixel value determination unit 38 determines a predetermined value as the pixel value included in the three - dimensional image for all pixels belonging to the pixel block associated with second - type individual acquisition management data 22b (S206). Here, for example, as the pixel value of the pixel, a pixel value representing black is determined.

[0101] Then, pixel value determination unit 38 confirms whether all individual acquisition management data 22 in this frame have undergone the processes of S203 to S206 (S207).

[0102] In the case where not all individual acquisition management data 22 in this frame have undergone the processes of S203 to S206 (S207: No), the process returns to the process of S202.

[0103] In the case where all individual acquisition management data 22 in this frame have undergone the processes of S203 to S206 (S207: Yes), image generation unit 40 generates a three - dimensional image of this frame (S208). Here, for example, a three - dimensional image is generated based on the pixel values determined in the process of S205 or S206.

[0104] Then, the display control unit 42 controls the display unit 18 to display the three-dimensional image generated in the process of S208 (S209), and the process returns to the process of S201.

[0105] In this processing example, the processes of S201 to S209 are repeatedly executed at a predetermined frame rate.

[0106] Note that, in the above processing example, the acquisition management data 20 in the frame may not be stored in the acquisition management data storage unit 30 at a predetermined frame rate. For example, the acquisition management data 20 common to all frames may be used. In this case, when the process of S209 ends, the process may return to the process of S202, and the process of the next frame may be executed.

[0107] Note that the present invention is not limited to the above embodiments.

[0108] In addition, the specific strings and numerical values in the above description and drawings are merely examples, and the present invention is not limited thereto.

Claims

1. An image generation device, comprising: a pixel value storage unit configured to store pixel values and alpha values; an acquisition management data storage unit configured to store acquisition management data, the acquisition management data being common to a plurality of frame images and indicating whether to acquire the pixel value of each pixel included in the frame image from the pixel value storage unit; an acquisition determination unit configured to, for the plurality of frame images, determine, based on the acquisition management data, pixels having pixel values acquired from the pixel value storage unit, and determine, for each pixel included in the frame image, whether to acquire the pixel value of the pixel from the pixel value storage unit; a pixel value acquisition unit configured to acquire, from the pixel value storage unit, the pixel values of pixels determined to have pixel values acquired from the pixel value storage unit; and a pixel value determination unit configured to determine the pixel value of each pixel, wherein the pixel value storage unit stores the pixel values of pixels determined to have pixel values acquired from the pixel value storage unit, the pixel value determination unit determines, for pixels determined to have pixel values acquired from the pixel value storage unit, the pixel value and alpha value acquired from the pixel value storage unit as the pixel value and alpha value of the pixels, the pixel value determination unit determines a predetermined value as the pixel value and alpha value of pixels determined to have pixel values not acquired from the pixel value storage unit; and the image generation device further includes an image generation unit configured to generate a superimposed image by superimposing a frame image having the pixel value and alpha value determined by the pixel value determination unit and another image on each other.

2. The image generation device according to claim 1, wherein the acquisition management data includes a plurality of individual acquisition management data, each piece of data being associated with a plurality of pixels, the acquisition determination unit determines, based on the individual acquisition management data, whether to acquire the pixel values of the plurality of pixels associated with the individual acquisition management data from the pixel value storage unit, and the pixel value determination unit determines a predetermined value as the pixel values of all pixels among the plurality of pixels associated with the individual acquisition management data and determined to have pixel values not acquired from the pixel value storage unit.

3. The image generation device according to claim 2, wherein, The individual acquisition management data is associated with a plurality of pixels occupying a rectangular area in the frame image.

4. The image generation device according to any one of claims 1 to 3, wherein, The acquisition management data indicates not to acquire the pixel values of pixels occupying a given area in the frame image from the pixel value storage unit.

5. An image generation method, comprising: a pixel value storage step of storing pixel values and alpha values in a pixel value storage unit; an acquisition management data storage step of storing acquisition management data in an acquisition management data storage unit, the acquisition management data being common to a plurality of frame images and indicating whether to acquire the pixel value of each pixel included in the frame image from the pixel value storage unit; An acquisition determination step for, for the plurality of frame images, determining, based on the acquisition management data, pixels having pixel values acquired from the pixel value storage unit, and determining, for each pixel included in the frame image, whether the pixel value of the pixel is acquired from the pixel value storage unit; A pixel value acquisition step for acquiring, from the pixel value storage unit, the pixel values of pixels determined to have pixel values acquired from the pixel value storage unit; And A pixel value determination step for determining the pixel value of each pixel, wherein, in the pixel value storage step, the pixel values of pixels determined to have pixel values acquired from the pixel value storage unit are stored in the pixel value storage unit, in the pixel value determination step, for pixels determined to have pixel values acquired from the pixel value storage unit, the pixel value and alpha value acquired from the pixel value storage unit are determined as the pixel value and alpha value of the pixel, and in the pixel value determination step, for pixels determined to have pixel values not acquired from the pixel value storage unit, a predetermined value is determined as the pixel value and alpha value of the pixel; The method further includes an image generation step for generating a superimposed image by superimposing a frame image having the pixel value and the alpha value determined in the pixel value determination step and another image on each other.

6. A computer-readable information storage medium having stored thereon a program for causing a computer to execute the following process: A pixel value storage process for storing a pixel value and an alpha value in a pixel value storage unit; An acquisition management data storage procedure is used to store acquisition management data in an acquisition management data storage unit. The acquisition management data is common to multiple frame images and indicates whether to acquire the pixel value of each pixel included in the frame image from the pixel value storage unit. Among them, The acquisition management data indicates that the pixel values of pixels occupying a given area in the frame image are not acquired from the pixel value storage unit; An acquisition determination process for, for the plurality of frame images, determining, based on the acquisition management data, pixels having pixel values acquired from the pixel value storage unit, and determining, for each pixel included in the frame image, whether the pixel value of the pixel is acquired from the pixel value storage unit; A pixel value acquisition process for acquiring, from the pixel value storage unit, the pixel values of pixels determined to have pixel values acquired from the pixel value storage unit; And A pixel value determination process for determining the pixel value of each pixel, wherein, in the pixel value storage process, the pixel values of pixels determined to have pixel values acquired from the pixel value storage unit are stored in the pixel value storage unit, in the pixel value determination process, for pixels determined to have pixel values acquired from the pixel value storage unit, the pixel value and alpha value acquired from the pixel value storage unit are determined as the pixel value and alpha value of the pixel, and in the pixel value determination process, for pixels determined to have pixel values not acquired from the pixel value storage unit, a predetermined value is determined as the pixel value and alpha value of the pixel; An image generation process for generating a superimposed image by superimposing a frame image having the pixel value and the alpha value determined in the pixel value determination process and another image on each other.

Images