Image generation device, image generation system, image generation method, and program

Abstract

To provide an image generation device that transmits alpha channel information further appropriately while suppressing an increase in the amount of data transmission.SOLUTION: An image generation device includes: acquisition means that acquires a first image where each pixel includes pixel data representing color and alpha channel representing transparency; determination means that groups the first image for N pixels (N>1) and determines a representative value of the alpha channel for each group; embedding means of 1) removing the alpha channel included in each pixel from the first image and 2) embedding, for each group, data of the representative value into some bits of the pixel data of N pixels included in the group; and transmission means that transmits a second image, which is an image obtained after the data of the representative value is embedded in the first image by the embedding means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image generation device, an image generation system, an image generation method, and a program. [Background technology]

[0002] In recent years, mixed reality (MR) technology and augmented reality (AR) technology have become known as technologies that seamlessly merge real space and virtual space in real time. There is also virtual reality (VR) technology, which reproduces only virtual space. One method for realizing these technologies is to use a head-mounted display (HMD). In this technology, the position and orientation of the HMD are estimated. Then, a computer graphics (CG) image, which is an image of the virtual space seen from the estimated position and orientation, is displayed so that the user can observe it.

[0003] Additionally, there are two methods for realizing MR and AR: the optical see-through method and the video see-through method. The optical see-through method is a method in which a virtual image that is viewed using a half mirror or the like is superimposed on the real space that is viewed directly with the naked eye. On the other hand, the video see-through method is a method in which a real image captured by a camera is superimposed on a rendered virtual image. Both methods have their own advantages and disadvantages. Therefore, it is common to use different applications to generate (output) the virtual reality depending on the method. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-095916 [Patent Document 2] Japanese Patent Application Laid-Open No. 2005-107780 Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, an image generating device generates CG and transmits RGBA (here, "A" is information related to transparency and is called "alpha channel") signals of the CG to an HMD. The HMD combines the real image with the virtual image by performing alpha blending of the CG and the real image. By using RGBA signals, the real image and the virtual image can be more appropriately combined. However, since RGBA signals contain more information than RGB signals, the transmission load is greater.

[0006] Patent Document 2 describes a method for reducing transmission load by including an alpha channel for each pixel of an RGB signal. However, this reduces the number of gradations in the RGB signal by the number of alpha channels. In addition, the number of gradations that can be included in the RGB signal is also limited.

[0007] Therefore, an object of the present invention is to provide a technique for more appropriately transmitting alpha channel information while suppressing an increase in the amount of data transmission. [Means for solving the problem]

[0008] One aspect of the present invention is a method for producing a medicament for the treatment of a pulmonary arthritis. an acquisition means for acquiring a first image in which each pixel includes pixel data representing a color and an alpha channel representing transparency; a determining means for grouping the first image into groups of N pixels (N>1) and determining a representative value of the alpha channel for each group; 1) removing the alpha channel included in each pixel from the first image; and 2) embedding means for each group, the data of the representative value into some bits of the pixel data of N pixels included in the group. a transmitting means for transmitting a second image, which is an image obtained after the representative value data is embedded in the first image by the embedding means; The image generating device is characterized by having:

[0009] One aspect of the present invention is a method for producing a medicament for the treatment of a pulmonary arthritis. an acquisition step of acquiring a first image in which each pixel includes pixel data representing color and an alpha channel representing transparency; a determining step of grouping N (N>1) pixels in the first image and determining a representative value of the alpha channel for each group; 1) removing the alpha channel included in each pixel from the first image; and 2) embedding, for each group, data of the representative value into some bits of the pixel data of N pixels included in that group; a transmitting step of transmitting a second image, which is an image obtained after the data of the representative value is embedded in the first image in the embedding step; The image generating method is characterized by comprising: [Effects of the Invention]

[0010] According to the present invention, it is possible to more appropriately transmit alpha channel information while suppressing an increase in the amount of data transmission. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a configuration diagram of an image generation system according to a first embodiment. [Figure 2] 10 is a flowchart showing the processing of an embedding unit according to the first embodiment. [Figure 3] 5 is a flowchart showing the processing of a separation unit according to the first embodiment. [Figure 4A] FIG. 2 is a diagram illustrating pixels before embedding according to the first embodiment. [Figure 4B] FIG. 4 is a diagram illustrating pixels after embedding according to the first embodiment. [Figure 4C] FIG. 4 is a diagram illustrating pixels after decoding according to the first embodiment. [Figure 5A] FIG. 10 is a diagram illustrating pixels before embedding according to the second embodiment. [Figure 5B]FIG. 10 is a diagram illustrating pixels after embedding according to the second embodiment. [Figure 5C] FIG. 10 is a diagram illustrating pixels after decoding according to the second embodiment. [Figure 6A] FIG. 11 is a diagram illustrating pixels before embedding according to the third embodiment. [Figure 6B] FIG. 10 is a diagram illustrating pixels after embedding according to the third embodiment. [Figure 6C] FIG. 11 is a diagram illustrating pixels after decoding according to the third embodiment. [Figure 7A] FIG. 10 is a diagram showing a list of embedding methods according to the fourth embodiment. [Figure 7B] FIG. 10 is a diagram illustrating embedding of a parameter ID according to the fourth embodiment. [Figure 8] 10 is a flowchart showing the processing of an embedding unit according to the fourth embodiment. [Figure 9] 10 is a flowchart showing the processing of a separation unit according to the fourth embodiment. [Figure 10A] FIG. 13 is a diagram illustrating pixels before embedding according to the fifth embodiment. [Figure 10B] FIG. 13 is a diagram illustrating pixels after embedding according to the fifth embodiment. [Figure 10C] FIG. 13 is a diagram illustrating pixels after decoding according to the fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0013] <Embodiment 1> An image generating device 100 according to a first embodiment will be described below with reference to Fig. 1. In the first embodiment, the image generating system 1 includes an image generating device 100 and an HMD 110 that are connected to each other. However, the HMD 110 may include some or all of the configuration (functions) of the image generating device 100.

[0014] (Configuration of the image generating device) The image generating device 100 includes a position and orientation estimation unit 101 , a CG image generating unit 102 , an embedding unit 103 , and an image transmitting unit 104 .

[0015] The position and orientation estimation unit 101 estimates at least one of the position and orientation of the HMD 110 (the user wearing the HMD 110) based on sensor data from the HMD 110. The estimation process may impose a large processing load. For this reason, in the first embodiment, the image generation device 100, which has a large amount of CPU and GPU resources, performs the estimation process.

[0016] The CG image generation unit 102 renders an image of the virtual space observed by the user as a CG image based on the estimation result by the position and orientation estimation unit 101 (the estimation result of at least one of the position and orientation of the HMD 110). In the first embodiment, the CG image is an image expressed in RGB values ​​and includes an alpha channel that is transparency information. In this case, the CG image generation unit 102 generates a CG image based on the estimation result by the position and orientation estimation unit 101, but may also generate an image that is unrelated to the position and orientation of the HMD 110.

[0017] The embedding unit (embedding unit) 103 embeds the alpha channel into the RGB signal by replacing a part of the RGB signal with the alpha channel. Details of the processing by the embedding unit 103 will be described later with reference to the flowchart in FIG. 2. The RGB signal has three pixel values ​​for each pixel of the image that indicate the intensity for expressing a color. The three pixel values ​​are the R value, G value, and B value in the RGB color space (RGB color system).

[0018] The image transmitting unit 104 transmits the RGB signal in which the alpha channel has been embedded by the embedding unit 103 to the HMD 110 as a new CG image.

[0019] (About the HMD configuration) The HMD 110 is a display device (information processing device) that can be worn on the head of a user. The HMD 110 includes a sensor unit 111, an imaging unit 112, an image processing unit 113, a synthesis unit 114, a display unit 115, a separation unit 116, and an image receiving unit 117.

[0020] The sensor unit 111 acquires sensor data for estimating at least one of the position and orientation of the HMD 110. The sensor unit 111 is generally configured with an acceleration sensor (a sensor that acquires the acceleration of the HMD 110 as sensor data) or an angular velocity sensor (a sensor that acquires the angular velocity of the HMD 110 as sensor data), etc. The sensor unit 111 may also use a sensor such as a geomagnetic sensor or a barometric pressure sensor.

[0021] Furthermore, the sensor unit 111 may be an image sensor that acquires an image of real space as sensor data. The position and orientation estimation unit 101 determines markers or feature points based on the image of real space acquired by the image sensor and performs calculation and comparison. This enables the position and orientation estimation unit 101 to estimate the position and orientation of the HMD 110 (user). Furthermore, the sensor unit 111 may be a sensor (a TOF sensor or a LiDAR sensor) that acquires depth information as sensor data by measuring the time it takes for irradiated light to return.

[0022] The imaging unit 112 acquires an image (real image) of the real space observed by the user. The image sensor 12 can be configured by combining, for example, an objective optical system with an image sensor. The image sensor is, for example, an image sensor that is compatible with color images and can acquire images with a wider angle of view than the angle of view of the display unit.

[0023] The image processing unit 113 performs development processing and various correction processing on the real image acquired by the imaging unit 112. As image sensors that support color images, sensors that require development processing using Bayer interpolation from RAW data are widely available. In addition, in order to improve the quality of the image, a wide variety of corrections are considered to be performed, such as shading correction to correct brightness, color correction, noise reduction, distortion correction, and angle of view correction for stereo cameras.

[0024] The composition unit 114 generates a composite image by performing alpha blending of the "RGB image representing the image of the virtual space acquired from the separation unit 116" and the "real image output by the image processing unit 113." Alpha blending achieves natural blending by using an A value (alpha value) representing transparency information (degree of transparency) in addition to each RGB pixel value (R value, G value, and B value). Therefore, in the real image, each pixel has pixel data indicating each RGB pixel value and an alpha channel indicating the alpha value. Here, in the composite image, the background of the CG image is made transparent based on the alpha value, so the real image acquired by the imaging unit 112 is used as the background image.

[0025] The display unit 115 displays the composite image. The display unit 115 may be an LCD (Liquid Crystal Display) or a micro OLED (Organic Light Emitting Diode). In addition, these devices are often combined with an eyepiece optical system. The user looks through the eyepiece optical system to observe the image synthesized by the synthesis unit 114.

[0026] The separator 116 acquires an alpha channel and an RGB image from the RGB signal acquired by the image receiver 117. Details of the processing by the separator 116 will be described later with reference to the flowchart of FIG.

[0027] The image receiving unit 117 transmits and receives CG images between the image generating device 100 and the HMD 110 .

[0028] In a typical video see-through HMD, RGBA signals must be transmitted and received in order for the synthesis unit 114 to synthesize a CG image and a real image. The transmission and reception of RGBA signals increases the amount of data transmitted, placing a heavy burden on the system. Meanwhile, in the first embodiment, an embedding unit 103 is added to the HMD 110, and a separation unit 116 is added to the HMD 110. This reduces the amount of data transmitted between the image transmission unit 104 and the image reception unit 117.

[0029] (Regarding the processing of embedded parts) The processing of the embedding unit 103 will be described with reference to the flowchart of FIG.

[0030] In step S201, the embedding unit 103 determines an alpha channel embedding method (substitution method). In the first embodiment, it is assumed that pixel values ​​are represented in big endian format. In this case, the embedding unit 103 determines the embedding method to be a method of grouping four adjacent pixels and embedding the average alpha channel of the group in the last bit (least significant bit) of each of the R pixel data and B pixel data. Note that if pixel values ​​are represented in little endian format, the embedding unit 103 determines the embedding method to be a method of embedding in the first bit (most significant bit) of each of the R pixel data and B pixel data.

[0031] In step S202, the embedding unit 103 acquires the CG image generated by the CG image generating unit .

[0032] In step S203, the embedding unit 103 reads pixel values ​​(RGBA pixel values) from the beginning of the CG image, and selects pixels for which no alpha channel has been embedded. Then, the embedding unit 103 groups N (N>1) pixels including the selected pixel. FIG. 4A is a diagram illustrating the grouped pixels. In FIG. 4A, four pixels 400, 410, 420, and 430 are treated as one group. Each pixel has R pixel data indicating an R value, G pixel data indicating a G value, B pixel data indicating a B value, and an alpha channel indicating an alpha value.

[0033] In step S204, the embedding unit 103 calculates the average value of the alpha values ​​of the alpha channels in the group. Then, the embedding unit 103 converts the average value into a binary number and divides it into individual bits. An overview of the processing in step S204 will be described with reference to FIG. 4A.

[0034] The alpha channels of pixels 400, 410, 420, and 430 are alpha channels 404, 414, 424, and 434. The embedding unit 103 calculates alpha channel 440, which indicates the average value of the alpha values ​​of alpha channels 404, 414, 424, and 434. Then, the embedding unit 103 determines alpha channel 440 as the representative value data of the group. In FIG. 4A, the value of alpha channel 440 is 255 in decimal notation, which is "11111111" when converted to binary. The embedding unit 103 generates divided data 441 to 448, which indicate numerical values ​​obtained by dividing this numerical value "11111111" into individual bits.

[0035] In step S205, the embedding unit 103 distributes and embeds each data segment among pixels in the group. In the first embodiment, the embedding unit 103 embeds the data segment into the last bit of each of the R pixel data and the B pixel data. At this time, the embedding unit 103 also deletes the alpha channel of each pixel from that pixel. An overview of the processing in step S205 will be described with reference to FIG. 4B.

[0036] Split data 441 is embedded as the value of the last bit of R pixel data 401 of pixel 400. Split data 442 is embedded as the value of the last bit of B pixel data 403. Similarly, when split data is embedded in pixels 410, 420, and 430, embedding of all 8 bits of data for alpha channel 440 is achieved. In this way, the embedding unit 103 does not embed multiple split data (two or more split data) into the same pixel data. Note that humans are more likely to notice changes in luminance than changes in color. Therefore, split data is not embedded (split data replacement is not performed) in G pixel data 402, 412, 422, and 432, which have the greatest impact on luminance. Furthermore, alpha channels 404, 414, 424, and 434 are deleted from pixels 400, 410, 420, and 430. As a result of the embedding, the embedding unit 103 can convert (compress) the RGBA signal into an RGB signal with an embedded alpha channel.

[0037] In step S206, embedding unit 103 determines whether the processes of steps S203 to S205 have been performed on all pixels of the CG image. If it is determined that the processes of steps S203 to S205 have been performed on all pixels of the CG image, the process proceeds to step S207. If it is determined that the processes of steps S203 to S205 have not been performed on at least some of the pixels of the CG image, the process proceeds to step S203 to perform the processes on pixels that have not been processed.

[0038] In step S207, the embedding unit 103 saves the RGB signal with the alpha channel embedded as a new CG image. The saved CG image (RGB signal) is then transmitted by the image transmitting unit 104 to the HMD 110 (image receiving unit 117).

[0039] (Regarding the processing of the separated part) The processing of the separator 116 will be described with reference to the flowchart of FIG.

[0040] In step S301, the separation unit 116 reads information about the embedding method for the alpha channel. In this way, the separation unit 116 acquires information about the embedding method determined by the embedding unit 103 in order to properly separate the alpha channel. Note that if the embedding unit 103 always uses the same embedding method, there is no need to specify a method for transmitting the embedding method information. On the other hand, if the embedding unit 103 changes the embedding method depending on the situation, it is also necessary to define a method for transmitting the embedding method information. In the first embodiment, a case where the same embedding method is always used will be described.

[0041] In step S302, the separation unit 116 acquires from the image receiving unit 117 the CG image (RGB signal) in which the alpha channel has been embedded.

[0042] In step S303, the separation unit 116 reads pixel values ​​(RGBA pixel values) from the beginning of the CG image, selects a pixel in a group for which the representative value of the alpha channel has not been restored, and then groups N (N>1) pixels including the selected pixel.

[0043] In step S304, the separator 116 obtains the data of the bits corresponding to the alpha channel from the group of pixels in the RGB pixel data in accordance with the loaded embedding method.

[0044] In step S305, the separator 116 combines the acquired bit data to restore the representative value of the alpha channel.

[0045] In step S306, the separation unit 116 sets the representative value of the restored alpha channel as the common alpha channel for all pixels in the group.

[0046] An overview of the processing of steps S303 to S306 will be described with reference to Figures 4B and 4C. Figure 4B is a diagram showing grouped pixels. Split data 441 to 448, each equivalent to a 1-bit value of the alpha channel, are embedded as the last bit values ​​of the R pixel data and B pixel data.

[0047] 4C shows how alpha channel 440 is restored by extracting and combining split data 441 to 448 embedded in the RGB signal. Alpha channel 440 is added as the alpha channel for each of pixels 400, 410, 420, and 430. For example, for pixel 400, alpha channel 440 is added in addition to R pixel data 401, G pixel data 402, and B pixel data 403. As a result, separation unit 116 can separate the alpha channel from the RGB signal and obtain (estimate) an RGBA signal.

[0048] Note that the pixel values ​​of the R pixel data 411 and the B pixel data 403, 413, and 433 change due to the embedding of the split data of the alpha channel. For example, the value of the R pixel data 411 is 250 before embedding as shown in FIG. 4A, but changes to 251 after embedding as shown in FIG. 4B. This change occurs when the suffix When embedding into 1 bit, a maximum of ±1 occurs. However, in the first embodiment, the change is treated as noise, and the subsequent restoration process is omitted.

[0049] In step S307, the separation unit 116 determines whether the processes of steps S303 to S306 have been performed on all pixels. If it is determined that the processes of steps S303 to S306 have been performed on all pixels, the process proceeds to step S308. By performing the processes of steps S303 to S306, the separation unit 116 can estimate the CG image generated by the CG image generation unit 102 (the CG image before the alpha channel is embedded by the embedding unit 103). If it is determined that the processes of steps S303 to S306 have not been performed on at least one pixel out of all pixels, the process proceeds to step S303.

[0050] In step S308, the separation unit 116 stores the estimated CG image as an RGBA signal. The stored CG image is then combined with the real image by the combination unit 114.

[0051] According to the first embodiment, an RGB signal (CG image) in which representative data of the alpha channel is embedded is transmitted, so the increase in the amount of data transmitted can be suppressed compared to when an RGBA signal is transmitted. Furthermore, the alpha channel can also be transmitted by transmitting an RGB signal.

[0052] <Embodiment 2> In the first embodiment, the image generation system 1 calculates the average value of the alpha values ​​of the alpha channels within a group of RGBA signals, divides it into bits, and embeds the average value into the R pixel data and B pixel data of each pixel. In the second embodiment, the image generation system 1 embeds the calculated average value concentratedly into specific pixels within the group, and interpolates the resulting changes in RGB values ​​based on the other pixels.

[0053] 5A and 5B are diagrams for explaining an outline of the processing performed by the embedding unit 103 in the second embodiment.

[0054] As shown in FIG. 5A, the alpha channels in a group are represented by alpha channels 504, 514, 524, and 534. The embedding unit 103 calculates alpha channel 540, which indicates the average value of the alpha values ​​of alpha channels 504, 514, 524, and 534, and determines the value of alpha channel 540 as the representative value of the group. The value of alpha channel 540 is 255 in decimal notation, which is "11111111" when converted to binary. The embedding unit 103 splits "11111111" bit by bit and embeds the first 3 bits of split data, 541, 542, and 543, into the RGB signal.

[0055] In the second embodiment, by limiting the number of bits used for embedding, the number of gradations in the alpha channel is reduced from 256 to 8. On the other hand, it is possible to reduce the impact on the R pixel data, G pixel data, and B pixel data.

[0056] 5B is a diagram showing how 3 bits of split data of alpha channel 540 are embedded only in pixel 500. Split data 541 is embedded in the last bit of R pixel data 501. Split data 542 is embedded in the last bit of G pixel data 502. Split data 543 is embedded in the last bit of B pixel data 503. Split values ​​are not embedded in pixels 510, 520, and 530.

[0057] FIG. 5C is a diagram illustrating an outline of the processing performed by the separator 116 in the second embodiment.

[0058] The separation unit 116 extracts the divided data 541 to 543 embedded in the pixel 500, The alpha channel 540 is restored (estimated) by combining the data according to the embedding method. Note that the separation unit 116 embeds the first divided data 541 in bits other than the first three bits. As a result, the reproducible alpha channel values ​​are eight levels: 255, 223, 191, 159, 96, 64, 32, and 0, but can also include the most commonly used values ​​of 255 and 0. The restored alpha channel 540 is added as the alpha channel of each of the pixels 500, 510, 520, and 530. For example, pixel 500 has the alpha channel 540 in addition to R pixel data 501, G pixel data 502, and B pixel data 503. As a result, the separation unit 116 can separate the alpha channel from the RGB signal and obtain (restore; estimate) an RGBA signal.

[0059] Note that the value of the B pixel data 503 was 96 before embedding as shown in FIG. 5A, but changed to 97 after embedding as shown in FIG. 5B. Although there was no change in the R pixel data 501 and the G pixel data 502 this time, their values ​​may change by up to ±1 in some cases. Therefore, the separation unit 116 attempts to restore the RGB values ​​of pixel 500 by calculating them through interpolation using pixels 510, 520, and 530. In the second embodiment, for example, the separation unit 116 calculates the average value of the B values ​​of all other pixels 510, 520, and 530 in the group and determines this average value as the B value of pixel 500 (the value of the B pixel data 503). In this way, the value of the B pixel data 503 (B value) can be restored to its original value of 96.

[0060] During interpolation, the accuracy of restoration can be improved by using surrounding pixels outside the group. Furthermore, the accuracy of restoration can be further improved by finding patterns in the arrangement of pixel values, such as vertical, horizontal, or diagonal lines, and performing interpolation according to these patterns.

[0061] According to the second embodiment, the representative value data of the alpha channel is embedded in only one pixel of a group of N pixels, preventing changes in the pixel values ​​of the other pixels, and thus allowing the HMD 110 to obtain a more accurate RGBA signal.

[0062] <Embodiment 3> In the first and second embodiments, the R, G, B, and A (alpha) values ​​of a CG image are each expressed in 8 bits. Recently, an increasing number of systems use signals with 8 or more bits to achieve richer gradation expression. However, increasing the number of signal bits increases the transmission volume, which is an issue. However, if the signal volume can be reduced by the third embodiment, it will be effective.

[0063] In the third embodiment, a case will be described in which each of the R, G, and B values ​​in an RGB signal is expressed in 10 bits. Furthermore, in the third embodiment, the virtual image output by the CG image generation unit 102 includes distance information in addition to the RGB image and alpha channel. The image generation device 100 also embeds this distance information in the RGB signal. In mixed reality, distance information is generally used to seamlessly blend the real image and the CG, including the context. Therefore, reducing the transmission volume for distance information is also an issue.

[0064] 6A and 6B are diagrams for explaining an outline of the processing of the embedding unit 103 in embodiment 3. In embodiment 3, the embedding unit 103 embeds not only the alpha channel but also distance information into the RGB signal.

[0065] 6A, R pixel data 601, G pixel data 602, and B pixel data 603 are each represented by 10 bits, while the alpha channel 604 and distance information 605 are each represented by 8 bits.

[0066] Therefore, the embedding unit 103 calculates an alpha channel 640 that indicates the average value of the alpha values ​​of the alpha channels 604, 614, 624, and 634 within the group, and determines the value of the alpha channel 640 as the representative value of the group. The value of the alpha channel 640 is 255 in decimal notation, which becomes "11111111" when converted to binary. The embedding unit 103 divides "11111111" into bits and embeds the divided data into the last two bits of each of the R pixel data and B pixel data.

[0067] The embedding unit 103 also calculates distance information 650, which indicates the average value of the distance information 605, 615, 625, and 635 within the group, and determines the value of distance information 650 as the representative value for the group. The value of distance information 650 is "36" in decimal notation, which becomes "00100100" when converted to binary. The embedding unit 103 divides "00100100" into bits and embeds the divided data into the last bit of each of the R pixel data and B pixel data.

[0068] 6B is a diagram showing the state in which split data has been embedded in an RGB signal. For example, at the end (least significant bit) of R pixel data 601, one bit of split data is embedded for the alpha channel and then distance information. Note that no split data is embedded in G pixel data, which has the greatest impact on luminance. As a result, luminance can be fully expressed with gradation expanded to 10 bits.

[0069] 6C is a diagram illustrating an overview of the processing performed by the separation unit 116 in the third embodiment. The separation unit 116 extracts the embedded split data from the pixels 600, 610, 620, and 630 and combines them according to the embedding method. In this way, the separation unit 116 restores the alpha channel 640 and distance information 650. The restored alpha channel 640 and distance information 650 are added as the alpha channel and distance information for the pixels 600, 610, 620, and 630, respectively. For example, in the case of pixel 600, the alpha channel 640 and distance information 650 are added in addition to the R pixel data 601, G pixel data 602, and B pixel data 603.

[0070] As a result, the separation unit 116 can separate the alpha channel and distance information from the RGB signal, and restore (estimate) the RGBA signal and distance information.

[0071] The alpha channel 604 and distance information 605 may also be expressed using any M bits (M>1), such as 6 bits or 10 bits. In this case, the representative value data of the alpha channel is divided into M pieces of divided data.

[0072] <Embodiment 4> In the first embodiment, the image generation system 1 always uses the same embedding method for the alpha channel. In the fourth embodiment, assuming that the embedding method may change, the image generation system 1 also embeds information indicating the embedding method in the RGB signal and transmits it.

[0073] FIG. 7A is a diagram showing an example of a list of embedding methods. In the list of embedding methods, a parameter ID is assigned to a combination of the number of pixels at the embedding destination, the embedding location, and the number of bits. For example, in the first embodiment, the number of pixels at the embedding destination is "4," i.e., "four pixels: pixels 400, 410, 420, and 430." The embedding location for pixel 400 is "RB" because it is R pixel data 401 and B pixel data 403. The number of bits of data to be embedded in one pixel data is "1." Therefore, when embedding is performed as in the first embodiment, referring to the list of embedding methods, the parameter ID corresponding to that embedding is "8." Therefore, the embedding unit 103 converts "8" into a binary number, resulting in "10" 00" is embedded in the RGB signal.

[0074] FIG. 7B shows how parameter IDs are embedded in pixels 400, 410, 420, and 430 in the group shown in FIG. 4A. Parameter ID 710, converted into a binary number, is divided into four pieces of data: 700, 701, 702, and 703. In pixel 400, an alpha channel has already been embedded in R pixel data 401 and B pixel data 403. Therefore, embedding unit 103 embeds divided data 700 at the end (least significant bit) of G pixel data 402. Similarly, embedding unit 103 embeds divided data 701 in G pixel data 412, divided data 702 in G pixel data 422, and divided data 703 in G pixel data 432. It should be noted that embedding unit 103 does not need to embed parameter IDs in all groups of a CG image; for example, it is sufficient to embed the parameter ID only in the first group.

[0075] Fig. 8 is a flowchart showing the processing of the embedding unit 103 according to the fourth embodiment. In the flowchart of Fig. 8, step S801 is added between step S205 and step S206 in the flowchart of Fig. 2. Note that in the steps having the same numbers as in Fig. 2, the same processing as that described in the first embodiment is executed.

[0076] In step S801, the embedding unit 103 embeds a parameter ID of an embedding method into pixels of the first group. Note that the group into which the parameter ID is embedded does not have to be the first group, but may be the last group, the second group, or the like.

[0077] Fig. 9 is a flowchart showing the processing of the separation unit 116 according to the fourth embodiment. Compared to the flowchart in Fig. 3, step S301 is deleted, and step S902 is added after step S302. Note that in the steps numbered the same as in Fig. 3, the same processing as that described in the first embodiment is executed.

[0078] In step S902, the separation unit 116 acquires the parameter ID of the embedding method embedded in the leading group. As a result, in a subsequent process, the separation unit 116 can restore the alpha channel by using the acquired parameter ID and referring to the list of embedding methods shown in Fig. 7A.

[0079] <Embodiment 5> In the first to third embodiments, the data format of the CG image is an RGB signal. On the other hand, the system may use a YUV signal, which is a signal that can further reduce the amount of transmission. A YUV signal has a Y value, which is the luminance component of image data, and U and V values, which are color components of the image data, in a YUV color space (YUV color system). Here, with a YUV signal, a format is used in which the resolution is reduced to 1 / 4 by thinning out every other pixel for the U and V values, which are color components whose changes are difficult for humans to notice. In a fifth embodiment, a case will be described in which the data format of the CG image generated by the CG image generation unit 102 is a YUVA signal.

[0080] FIG. 10A shows four pixels grouped into one group. Pixels 1000, 1010, 1020, and 1030 are treated as one group. The alpha channels in the group are alpha channels 1003, 1013, The embedding unit 103 generates an alpha channel 1040 that indicates the average value of the alpha values ​​of the alpha channels 1003, 1013, 1023, and 1033. The embedding unit 103 calculates the value of the alpha channel 1040 and determines it as the representative value of the group. The value of the alpha channel 1040 is 255 in decimal notation, which is "11111111" when converted to binary notation. The embedding unit 103 divides "11111111" into bits. Then, 4 bits of split data 1041, 1042, 1043, and 1044 from the first bit are embedded into the YUV pixel data. Note that by limiting the number of bits, the number of gradations in the alpha channel is reduced from 256 to 16.

[0081] FIG. 10B shows how these 4-bit divided data 1041, 1042, 1043, and 1044 are embedded in the last bit of each of the U pixel data 1002 and 1032 and the V pixel data 1012 and 1022.

[0082] 10C is a diagram showing how the alpha channel is separated from a YUV signal in which the alpha channel is embedded. The separation unit 116 extracts the divided data 1041 to 1044 embedded in the UV pixel data 1002, 1012, 1022, and 1032 and combines them according to a predetermined embedding method. In this way, the separation unit 116 restores the alpha channel 1040. Note that the separation unit 116 fills all bits of the alpha channel 1040 except for the first four bits with the first divided data 1041. This allows the reproduced alpha channel to include the most commonly used values ​​255 and 0.

[0083] The restored alpha channel 1040 is added as the alpha channel for each of the pixels 1000, 1010, 1020, and 1030. For example, in pixel 1000, the alpha channel 1040 is added in addition to the Y pixel data 1001 and U pixel data 1002. As a result, the separation unit 116 can separate the alpha channel from the YUV signal and restore the YUV signal and the alpha channel.

[0084] Furthermore, in the above, "If A is greater than or equal to B, proceed to step S1; if A is less than (lower than) B, proceed to step S2" may be read as "If A is greater than (higher than) B, proceed to step S1; if A is less than or equal to B, proceed to step S2." Conversely, "If A is greater than (higher than) B, proceed to step S1; if A is less than (lower than) B, proceed to step S2" may be read as "If A is greater than (higher than) B, proceed to step S1; if A is less than (lower than) B, proceed to step S2." Therefore, unless a contradiction arises, "greater than or equal to A" may be read as "greater than (higher; longer; more) than A," and "less than or equal to A" may be read as "less than (lower; shorter; fewer) than A." Furthermore, "greater than (higher; longer; more) than A" may be read as "greater than or equal to A," and "less than (lower; shorter; fewer) than A" may be read as "less than or equal to A."

[0085] The various controls described above may or may not be performed by a single piece of hardware (e.g., a processor or circuit). The entire device may be controlled by multiple pieces of hardware (e.g., multiple processors, multiple circuits, or a combination of one or more processors and one or more circuits) sharing the processing.

[0086] The above processor is a processor in the broad sense, and includes general-purpose processors and dedicated processors. General-purpose processors include, for example, CPUs (Central Processing Units), MPUs (Micro Processing Units), and DSPs (Digital Signal Processors). Dedicated processors include, for example, GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), and PLDs (Programmable Logic Devices). Programmable logic devices include, for example, FPGAs (Field Programmable Gate Arrays) and CPLDs (Complex Programmable Logic Devices).

[0087] Furthermore, although the embodiments of the present invention have been described in detail, the present invention is not limited to these specific embodiments. Furthermore, the above-described embodiments merely represent one embodiment of the present invention, and various embodiments can be combined as appropriate.

[0088] <Other embodiments> The present invention can also be realized by a process in which a program that realizes one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device read and execute the program, or by a circuit that realizes one or more functions.

[0089] The disclosure of the above embodiments includes the following configurations, methods, and programs. (Configuration 1) an acquisition means for acquiring a first image in which each pixel includes pixel data representing a color and an alpha channel representing transparency; a determining means for grouping the first image into groups of N pixels (N>1) and determining a representative value of the alpha channel for each group; 1) removing the alpha channel included in each pixel from the first image; and 2) embedding means for each group, the data of the representative value into some bits of the pixel data of N pixels included in the group. a transmitting means for transmitting a second image, which is an image obtained after the representative value data is embedded in the first image by the embedding means; An image generating device comprising: (Configuration 2) the representative value is an M-bit (M>1) value, the embedding means embeds each of two or more pieces of divided data among the M pieces of divided data obtained by dividing the representative value for each bit into some bits of pixel data of the N pixels in the group corresponding to the representative value; 2. The image generating device according to configuration 1, (Configuration 3) the embedding means embeds the two or more divided data into pixel data of one pixel among the N pixels of the group corresponding to the representative value; 3. The image generating device according to configuration 2. (Configuration 4) the pixel data representing the color includes a plurality of pixel values ​​indicating intensities for representing the color; the embedding means does not embed any of the M pieces of divided data into pixel values ​​that have the greatest influence on luminance among the plurality of pixel values. 3. The image generating device according to configuration 2. (Configuration 5) the plurality of pixel values ​​are R values, G values, and B values ​​in an RGB color space; the embedding means does not embed any of the M pieces of divided data into a G value. 5. The image generating device according to configuration 4. (Configuration 6) the plurality of pixel values ​​include at least a Y value among Y values, U values, and V values ​​in a YUV color space; The embedding means does not embed any of the M pieces of divided data into a Y value. 5. The image generating device according to configuration 4. (Configuration 7) the embedding means embeds the representative value data into some bits of the pixel data of the N pixels so that two or more pieces of divided data are not embedded in the same pixel value in the pixel data of the N pixels; 7. The image generating device according to any one of configurations 4 to 6. (Configuration 8) When the pixel value is expressed in big endian, the embedding means embeds the divided data into the least significant bit of the pixel value. 8. The image generating device according to any one of configurations 4 to 7. (Configuration 9) When the pixel value is expressed in little endian, the embedding means embeds the divided data into the most significant bit of the pixel value. 8. The image generating device according to any one of configurations 4 to 7. (Configuration 10) the embedding means embeds a value representing distance information for each group into some bits of the pixel data of the N pixels included in the group; 10. The image generating device according to any one of configurations 1 to 9. (Configuration 11) the embedding means embeds data indicating a method for embedding the representative value data into some bits of the pixel data of N pixels included in any one of the groups of the first image. 11. The image generating device according to any one of configurations 1 to 10. (Configuration 12) an image generating device according to any one of configurations 1 to 11; an information processing device that acquires the second image from the image generating device; and the information processing device has a separation means for acquiring the representative value of the alpha channel from the second image; An image generation system comprising: (Configuration 13) the separating means estimates the first image based on the second image; 13. The image generating system according to claim 12. (Configuration 14) the information processing device further comprises a synthesis means for generating an image by synthesizing a third image, each pixel of which includes pixel data representing a color and an alpha channel representing transparency, with the first image estimated by the separation means; 14. The image generating system according to claim 13, (method) an acquisition step of acquiring a first image in which each pixel includes pixel data representing color and an alpha channel representing transparency; a determining step of grouping N (N>1) pixels in the first image and determining a representative value of the alpha channel for each group; 1) removing the alpha channel included in each pixel from the first image; and 2) embedding, for each group, data of the representative value into some bits of the pixel data of N pixels included in that group; a transmitting step of transmitting a second image, which is an image obtained after the data of the representative value is embedded in the first image in the embedding step; An image generating method comprising: (program) 12. A program for causing a computer to function as each means of the image generating device according to any one of configurations 1 to 11. [Explanation of symbols]

[0090] 100: Image generating device, 102: CG image generating unit (acquisition unit), 103: Embedding unit (determining unit), 104: Image transmitting unit

Claims

1. an acquisition means for acquiring a first image in which each pixel includes pixel data representing a color and an alpha channel representing transparency; a determining means for grouping the first image into groups of N pixels (N>1) and determining a representative value of the alpha channel for each group; 1) removing the alpha channel included in each pixel from the first image; and 2) embedding means for each group, the data of the representative value into some bits of the pixel data of N pixels included in the group. a transmitting means for transmitting a second image, which is an image obtained after the representative value data is embedded in the first image by the embedding means; An image generating device comprising:

2. the representative value is an M-bit (M>1) value, the embedding means embeds each of two or more pieces of divided data out of M pieces of divided data obtained by dividing the representative value for each bit into some bits of pixel data of N pixels in a group corresponding to the representative value; 2. The image generating device according to claim 1.

3. the embedding means embeds the two or more divided data into pixel data of one pixel among the N pixels of the group corresponding to the representative value; 3. The image generating device according to claim 2.

4. the pixel data representing the color includes a plurality of pixel values ​​indicating intensities for representing the color; the embedding means does not embed any of the M pieces of divided data into pixel values ​​that have the greatest influence on luminance among the plurality of pixel values.

3. The image generating device according to claim 2.

5. the plurality of pixel values ​​are R values, G values, and B values ​​in an RGB color space; the embedding means does not embed any of the M pieces of divided data into a G value; 5. The image generating device according to claim 4.

6. the plurality of pixel values ​​include at least a Y value among Y values, U values, and V values ​​in a YUV color space; the embedding means does not embed any of the M pieces of divided data into the Y value; 5. The image generating device according to claim 4.

7. the embedding means embeds the representative value data into some bits of the pixel data of the N pixels so that two or more divided data are not embedded in the same pixel value in the pixel data of the N pixels.

5. The image generating device according to claim 4.

8. When the pixel value is expressed in big endian, the embedding means embeds the divided data into the least significant bit of the pixel value.

5. The image generating device according to claim 4.

9. When the pixel value is expressed in little endian, the embedding means embeds the divided data into the most significant bit of the pixel value.

5. The image generating device according to claim 4.

10. the embedding means embeds a value representing distance information for each group into some bits of the pixel data of N pixels included in the group; 2. The image generating device according to claim 1.

11. the embedding means embeds data indicating a method for embedding the representative value data into some bits of the pixel data of N pixels included in any one of the groups of the first image; 2. The image generating device according to claim 1.

12. An image generating device according to any one of claims 1 to 11; an information processing device that acquires the second image from the image generating device; and the information processing device has a separation means for acquiring the representative value of the alpha channel from the second image; An image generation system comprising:

13. the separating means estimates the first image based on the second image.

13. The image generation system according to claim 12.

14. the information processing device further comprises a synthesis means for generating an image by synthesizing a third image, each pixel of which includes pixel data representing a color and an alpha channel representing transparency, with the first image estimated by the separation means; 14. The image generation system according to claim 13.

15. an acquisition step of acquiring a first image in which each pixel includes pixel data representing color and an alpha channel representing transparency; a determining step of grouping the first image into groups of N pixels (N>1) and determining a representative value of the alpha channel for each group; 1) removing the alpha channel included in each pixel from the first image; and 2) embedding, for each group, data of the representative value into some bits of the pixel data of N pixels included in that group. a transmitting step of transmitting a second image, which is an image obtained after the representative value data is embedded in the first image in the embedding step; An image generating method comprising:

16. A program for causing a computer to function as each of the means of the image generating apparatus according to any one of claims 1 to 11.

Images