Video decoding device, diffraction computation device, object light decoding device, object light decoding method, and program

Abstract

To provide a video decoding device which enables application of object light required for holographic three-dimensional image display to existing video encoding standards and allows the object light to be efficiently encoded, transferred, and correctly displayed on a receiving device.SOLUTION: A diffraction computation device 40 of the present invention uses object light data decoded as a video image and information indicative of a relationship between a recording surface of the object light data and a display surface to convert the object light data into object light data on the display surface.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] The present invention relates to a moving image decoding device, an analysis calculation device, an object light decoding device, an object light decoding method, and a program.

Background Art

[0002] In Non-Patent Document 1, a technique related to video image encoding is disclosed. Also, in Non-Patent Document 2, a technique related to wavelength correction is disclosed.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] The data targeted by Non-Patent Document 1 is mainly an image that is premised on being displayed on a device (hereinafter referred to as a display) that displays images and videos by emitting light from pixels of three colors (RGB) such as a liquid crystal display.

[0005] On the other hand, for stereoscopic video display called holography, data on the amplitude and phase of object light is required, and due to the different formats, there has been a problem that conventional image encoding techniques cannot be simply applied for encoding.

[0006] In addition, for the amplitude-phase data of light, conversion of the interval according to the playback optical system (device) is necessary.

[0007] Here, among the data for holographic display, the "amplitude-phase distribution of light from an object that does not depend on the playback optical system (device)", which is the object of the present invention, will be referred to as "object light" in this embodiment.

[0008] Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to provide a moving image decoding device, a diffraction calculation device, an object light decoding device, an object light decoding method, and a program that can apply object light necessary for three-dimensional video display of holography to an existing moving image encoding standard, efficiently encode and transmit the object light, and correctly display it on a receiving device.

Means for Solving the Problems

[0009] A first feature of the present invention is a moving image decoding device including a decoding unit that decodes object light data converted into a moving image from a bit stream, and a metadata processing unit that reads metadata, and the metadata processing unit reads, as a part of the metadata, information indicating the relationship between the recording surface and the display surface of the object light data.

[0010] A second feature of the present invention is a diffraction calculation device that converts the decoded object light data into object light data on the display surface based on the object light data decoded as a moving image and information indicating the relationship between the recording surface and the display surface of the object light data.

[0011] A third feature of the present invention is an object light decoding device including a moving image decoding device, a diffraction calculation device, and an object light correction device. The moving image decoding device includes a decoding unit that decodes object light data converted into a moving image from a bit stream, and a metadata processing unit that reads metadata. The metadata processing unit reads, as part of the metadata, information indicating the relationship between the recording surface and the display surface of the object light data. The diffraction calculation device converts the decoded object light data into object light data on the display surface based on the object light data decoded as a moving image and the information indicating the relationship between the recording surface and the display surface of the object light data. The object light correction device corrects the object light data on the display surface and outputs object light.

[0012] A fourth feature of the present invention is an object light decoding method, including a step of decoding object light data converted into a moving image from a bit stream, a step of reading, as part of metadata, information indicating the relationship between the recording surface and the display surface of the object light data, a step of converting the decoded object light data into object light data on the display surface based on the decoded object light data and the information indicating the relationship between the recording surface and the display surface of the object light data, and a step of correcting the object light data on the display surface and outputting object light.

[0013] A fifth feature of the present invention is a program that causes a computer to function as a diffraction calculation device. The diffraction calculation device converts the decoded object light data into object light data on the display surface based on the object light data decoded as a moving image and the information indicating the relationship between the recording surface and the display surface of the object light data.

[0014] A sixth feature of the present invention is a program that causes a computer to function as an object light decoding device including a moving image decoding device, a diffraction calculation device, and an object light correction device. The moving image decoding device includes a decoding unit that decodes object light data converted into a moving image from a bit stream, and a metadata processing unit that reads metadata. The metadata processing unit reads, as part of the metadata, information indicating the relationship between the recording surface and the display surface of the object light data. The diffraction calculation device converts the decoded object light data into object light data on the display surface based on the object light data decoded as a moving image and the information indicating the relationship between the recording surface and the display surface of the object light data. The object light correction device corrects the object light on the display surface and outputs the object light.

Advantages of the Invention

[0015] According to the present invention, it is possible to apply object light necessary for holographic stereoscopic video display to an existing moving image encoding standard, efficiently encode and transmit the object light, and correctly display it on a receiving device. It is possible to provide a moving image decoding device, a diffraction calculation device, an object light decoding device, an object light decoding method, and a program.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

[0017] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0018] <First Embodiment> Hereinafter, with reference to FIG. 1, the object light decoding device 1 according to an embodiment of the present invention will be described.

[0019] As shown in FIG. 1, the object light decoding device 1 according to the present embodiment includes a moving image decoding device 10, an object light inverse quantization device 20, and an object light correction device 30.

[0020] (Moving Image Decoding Device 10) The moving image decoding device 10 is configured to take as input a bitstream (encoded bitstream) obtained by encoding object light and output integerized object light data (n-bit int).

[0021] The moving image decoding device 10 is configured to generate integerized object light data (n-bit int) by decoding a bitstream (encoded bitstream) with metadata attached as SEI (Supplemental Enhancement Information) or VUI (Video Usability Information) supported by Non-Patent Document 1 or the like.

[0022] Note that the encoded bitstream input to the moving image decoding device 10 is as follows.

[0023] The object light data used for holographic reproduction is two-dimensionally distributed amplitude-phase data. For example, it is assumed that a two-dimensional array in which amplitude data and phase data for each RGB color channel (or real part data and imaginary part data when the amplitude data and phase data are converted into values on the complex plane) are recorded as floating-point or fixed-point variables for each sampling point.

[0024] By performing quantization (integerization, imaging) on such object light data, it can be treated in the same way as general image data.

[0025] Therefore, in the present embodiment, it is assumed that a bitstream (encoded bitstream) with the metadata described in FIG. 2 attached is input after encoding the object light data quantized as described above.

[0026] That is, it can also be said that the moving image decoding device 10 is configured to receive an encoded bitstream and decode image data.

[0027] Specifically, the moving image decoding device 10 obtains quantized object light data by performing inverse DCT and decoding based on prediction residuals.

[0028] The format of the object light data obtained at this time is the same as the format of the object light data input to a moving image encoding device (not shown).

[0029] For example, when the moving image encoding device generates and encodes image data (object light data) having integer values of 0 to 255 for each color channel from object light, the object light data (decoded quantized object light signal) obtained by the moving image decoding device 10 also has the same format.

[0030] Here, regarding the object light data, when the above-mentioned amplitude data and phase data are converted into values on the complex plane, the real part data and the imaginary part data may each be similarly a two-dimensional array of floating point numbers. Hereinafter, particularly when there is no description limited to the above-mentioned amplitude data and phase data, it may be treated as a complex number composed of such real part data and imaginary part data.

[0031] Note that such a bit stream may be transmitted via a transmission path. Alternatively, such a bit stream may be stored in a storage medium and then provided from a moving image encoding device (not shown) to the moving image decoding device 10.

[0032] (Object light inverse quantization device 20) The object light inverse quantization device 20 is configured to output object light data (float) described by a decimal point by applying linear or non-linear inverse quantization to each of the amplitude and phase with respect to the object light data (n bit int) output from the moving image decoding device 10.

[0033] The object light inverse quantization device 20 is configured to convert the quantized (integerized) object light data into an amplitude-phase format necessary for hologram reproduction by performing inverse quantization.

[0034] Specifically, as shown in FIG. 3, the object light inverse quantization device 20 includes an object light data determination unit 21, an inverse quantization mode selection unit 22, and a mapping unit 23.

[0035] The object light data determination unit 21 is configured to determine whether the encoded bitstream input to the moving image decoding device 10 encodes object light.

[0036] When the inverse quantization mode selection unit 22 determines that the encoded bitstream encodes object light by the object light data determination unit 21, it determines what quantization was applied to the object light data that was floating-point data, and is configured to select an inverse quantization mode corresponding to such a determination result.

[0037] The mapping unit 23 is configured to perform mapping of object light data (n bit int) to object light data (float) based on the inverse quantization mode selected by the inverse quantization mode selection unit 22.

[0038] (Object light correction device 30) The object light correction device 30 is configured to correct the object light data (float) output from the object light inverse quantization device 20 according to the pixel interval of the reproduction optical system.

[0039] The specific functions of the object light correction device 30 will be described later.

[0040] (Operation of the object light decoding device 1) Hereinafter, an example of the operation of the object light decoding device 1 according to the present embodiment will be described with reference to FIGS. 4 to 7.

[0041] As shown in FIG. 4, in step S101, the moving image decoding device 10 decodes the integerized object light data (n bit int) from the encoded bitstream acquired from the object light encoding device.

[0042] In steps S103 to S106, the object light inverse quantization device 20 converts the integerized object light data (n-bit int) into object light data (float) described by a decimal point such as a floating point number.

[0043] That is, the object light inverse quantization device 20 performs mapping to a value close to the original object light data using the metadata in the encoded bitstream processed by the moving image decoding device 10.

[0044] Specifically, in step S103, the object light inverse quantization device 20 refers to object_wave_flag in the metadata and determines whether the encoded bitstream encodes object light.

[0045] If it is determined that the encoded bitstream does not encode object light (object_wave_flag = 0), the object light inverse quantization device 20 performs processing according to the procedure defined in Non-Patent Document 1 or the like as another data format, and this operation ends.

[0046] If it is determined that the encoded bitstream encodes object light (object_wave_flag = 1), in steps S104 and S105, the object light inverse quantization device 20 converts the object light data (n-bit int) generated by the moving image decoding device 10 into data in amplitude-phase format so as to be available for holographic stereoscopic video display.

[0047] That is, in step S104, the object light inverse quantization device 20 refers to data_type in the metadata and determines what kind of data the object light data (n-bit int) decoded from the encoded bitstream consists of.

[0048] Specifically, as shown in FIG. 5, the object light inverse quantization device 20 determines which of the amplitude data (data_type = 0), phase data (data_type = 1), real part data (data_type = 2), imaginary part data (data_type = 3), amplitude-phase combined data (data_type = 4), and real part-imaginary part combined data (data_type = 5) the object light data (n-bit int) corresponds to according to the value of data_type.

[0049] Here, for actual holographic playback, a set of amplitude data and phase data (or real part data and imaginary part data) is required. However, since it is also assumed that the amplitude data and phase data are transmitted in separate streams, such a determination is made.

[0050] Note that as an effect of treating the amplitude data and phase data as separate streams, it is possible to individually apply processing specialized for each of the amplitude data and phase data. On the other hand, at the time of playback, it is necessary to transmit each of the amplitude data and phase data individually, and it is assumed that the transmission efficiency will decrease.

[0051] On the contrary, by treating the amplitude data and phase data as amplitude-phase combined data, it is possible to transmit in a single session when transmitting the bit stream, and in addition, an effect of suppressing the total bit rate can be expected.

[0052] Also, when data_type is amplitude-phase combined data (4) or real part-imaginary part combined data (5), that is, when a set of amplitude data and phase data (real part data and imaginary part data) is included in one bit stream, the object light inverse quantization device 20 needs to determine which bits are the amplitude (real part), and such a determination is made by the parameter amp_real_bit_param.

[0053] When it is determined that the data_type is real part data (2), imaginary part data (3), or real and imaginary combined data (5), the object light inverse quantization device 20 performs conversion processing to data in amplitude-phase format.

[0054] In step S106, the object light inverse quantization device 20 determines what quantization has been applied to the object light data described in floating point, and selects an inverse quantization mode corresponding to such determination result.

[0055] Here, when the mapping_type indicating such inverse quantization mode (mapping method) indicates linear inverse quantization (mapping_type = 0), based on the object_wave_max indicating the maximum value of the object light data included in the metadata, the object_wave_min indicating the minimum value of the object light data, and the number of bits, the value range of the object light data is equally divided, and the representative value included in each divided section is mapped as the value of the quantized sampling point.

[0056] Examples of the representative value include the average value, maximum value, minimum value, etc. of the divided section, and such representative value is specified by the representative_value_type.

[0057] On the other hand, when the mapping_type indicating such inverse quantization mode (mapping method) indicates non-linear inverse quantization (mapping_type = 1), similarly, based on the mapping_table indicating the mapping table (see FIG. 6) included in the metadata and the transfer_characteristics in the VUI, mapping is performed to the value of the object light data.

[0058] Specifically, in step S106, the object light inverse quantization device 20 uses the mapping table to perform conversion to the mapping value corresponding to the pixel value of the integerized object light data, that is, conversion from integer to floating point.

[0059] Similarly, when using the transfer function of transfer_characteristics or the like, the object light inverse quantization device 20 may perform mapping to the function output using each pixel value as an input. At this time, it should be noted that the output range is compressed and expanded to a range between the maximum value of the object light data described in object_wave_max and the minimum value of the object light data described in object_wave_min.

[0060] In step S107, the object light correction device 30 corrects the sampling interval of the object light data with reference to the metadata according to the characteristics of the receiving device (display device).

[0061] Here, the object light data (float) obtained by the object light inverse quantization device 20 is data that does not depend on the reproduction optical system of the receiving device, and usually cannot be correctly reproduced as a desired holography as it is.

[0062] Therefore, the object light correction device 30 corrects the object light data (float) to match the reproduction optical system of the receiving device, enabling reproduction.

[0063] In holography reproduction, the physical distance between adjacent pixels of the display device is an important parameter that affects the quality of the reproduced image.

[0064] Therefore, usually, when calculating the object light data, the physical distance between sampling points is set so that a desired holography reproduced image can be obtained in a virtual 3D space on a computer.

[0065] The inconsistency between the sampling interval set here and the pixel interval of the display device (display) becomes a factor in the deterioration of the final reproduced image.

[0066] Assuming that the distance interval between sampling points during the calculation of the object light data is given as pixel_pitch in the metadata of the encoded bitstream, the object light correction device 30 corrects the sampling interval of the object light data.

[0067] FIG. 7 shows an example of a method for correcting such a sampling interval.

[0068] In the example of FIG. 7(b), consider displaying, on a display device with 8K resolution / pixel pitch = 2 nm, a hologram obtained from object light data calculated with 8K resolution / pixel pitch = 4 nm.

[0069] At this time, if the hologram is displayed as it is, the 8K resolution / pixel pitch will become 2 nm, and there is a possibility that the reproduced image will be different from the intention.

[0070] Therefore, the object light correction device 30 corrects the object light data in advance so that the sampling interval becomes 2 nm and then displays it.

[0071] That is, in the above example, the object light correction device 30 arranges new sampling points in the middle of the sampling points, performs interpolation so that the sampling interval becomes 2 nm, and then displays the corrected object light data, thereby solving the problem during reproduction.

[0072] Such upsampling processing can widely apply existing interpolation techniques such as bicubic, bilinear, and nearest neighbor.

[0073] The object light correction device 30 may also perform interpolation so that the difference from adjacent pixels becomes large. In such a case, the object light correction device 30 may perform interpolation so that the total variation of all or a part of the object light data is maximized.

[0074] Similarly, when the pixel pitch of the display device is large with respect to the sampling interval during the calculation of the object light data shown in FIG. 7(a), the object light correction device 30 may downsample the object light data along the pixel pitch of the display device and then display such object light data on the display device.

[0075] Due to the above interpolation process, there may be a difference between the number of pixels of the object light data and the number of pixels of the display device when displaying on the display device.

[0076] When the number of pixels of the object light data is large, as shown in FIG. 7(c), the display device may perform display using only a part of the object light data.

[0077] In such a case, regarding which area to display on the display device, the center of the object light data may be selected, or the position facing the object may be selected based on the position of the object given in advance as metadata, or the RGB representative value of the image captured by another imaging system of the image actually reproduced from the object light data may be selected. The position where the value is maximum may be selected.

[0078] Also, when the number of pixels of the object light data is small, all the object light data may be displayed using a partial area of the display device (an area that is not used in a part of the display device is generated).

[0079] As a method for selecting the display position of the object light data in such a case, it may be simply displayed at the center of the display device, or it may be displayed at the position closest thereto according to a predetermined viewing position or eye position of the user.

[0080] According to the above-described embodiment, the object light necessary for holographic stereoscopic video display can be applied to an existing moving image coding standard, and after efficiently coding and transmitting the object light, it can be correctly displayed on the receiving device.

[0081] (Modification Example 1) Hereinafter, Modification Example 1 of the above-described First Embodiment will be mainly described with respect to the differences from the above-described First Embodiment.

[0082] In the above-described First Embodiment, the object light correction device 30 only corrects the sampling interval.

[0083] On the other hand, in the first modification example, as shown in FIG. 10, on the premise that the wavelength of the object light data and the distance from the display device to the recorded object surface are given in the metadata of the encoded bit stream as wavelength_R, wavelength_G, wavelength_B, and object_depth, respectively, the wavelength correction unit 32 of the object light correction device 30 corrects the wavelength shift between the transmission device and the reception device, and reduces the color bleeding of the reproduced image caused by such a wavelength shift.

[0084] As shown in FIG. 10, the wavelength correction unit 32 includes a correction calculation unit 33 that performs wavelength correction calculation by a procedure shown in Non-Patent Document 2 or the like, a correction evaluation unit 34 for evaluating the correction result, and an object distance recording unit 35.

[0085] For example, in the wavelength correction according to Non-Patent Document 2, since correction is applied so that color bleeding is eliminated at the object depth from the display device to the object surface, color bleeding may occur when the distance is away from the object depth used in the correction calculation.

[0086] Therefore, as shown in step S109 of FIG. 4, the correction evaluation unit 34 evaluates the reproduced image of the object light data (float) in order to correct the value of object_depth so that the degree of color bleeding of the reproduced image becomes small.

[0087] The correction evaluation unit 34 outputs a reproduced image from the object light data (float) corrected by the correction calculation unit 33 and evaluates the quality.

[0088] Since the color bleeding of the reproduced image caused by the wavelength shift is caused by the shift of the contour portions of the respective color signals, by correcting the wavelength, the contour portions of the respective color signals coincide, and the correlation of the color signals increases.

[0089] Therefore, the improvement in the quality of the reproduced image by wavelength correction can be quantitatively evaluated by measuring the color signals.

[0090] The correction evaluation unit 34 may calculate the correlation of color signals based on the difference value, covariance, correlation coefficient, etc. between color signals.

[0091] For example, the correction evaluation unit 34 uses a local search method such as the hill climbing method with the correlation of color signals of the entire reproduced image or the contour part of the reproduced image as an evaluation function to search for a correction value of object_depth at which the evaluation function becomes maximum.

[0092] For the detection of the contour part of the reproduced image, the Canny method or a second-order differential filter may be used.

[0093] The reproduced image may be rendered by computer simulation based on object light data (float) or interference fringes obtained from object light data (float), or may be image data obtained by imaging a reproduced image in which interference fringes are displayed on a display device (display).

[0094] In addition to automatically optimizing the object distance as described above, the correction evaluation unit 34 may manually set the object distance while visually observing the reproduced image displayed on the display device (display).

[0095] The value of object_depth corrected using the correction evaluation unit 34 is stored in the object distance recording unit 35 and input to the correction calculation unit 33.

[0096] Also, in the above-described first embodiment, the metadata and the object light data are associated in a one-to-one correspondence. For example, when decoding a bitstream composed of a plurality of object light data continuous in the time direction, metadata must be given to all the object light data constituting the bitstream.

[0097] In contrast, this modification example 1 exemplifies a case where the metadata attached to the decoded object light data (n-bit int) is associated with the object light data (n-bit int) output later in the output order of the moving image decoding device 10.

[0098] As shown in FIG. 9, the moving image decoding apparatus 10 according to the first modification example includes a decoding unit 11 that decodes an encoded bit stream and a metadata processing unit 12.

[0099] The decoding unit 11 is configured to decode object light data that has been converted into a moving image from the received bit stream.

[0100] As shown in FIG. 8, when referring to information necessary for processing the object light data (n-bit int) output from the moving image decoding apparatus 10, the object light inverse quantization apparatus 20 and the object light correction apparatus 30 according to the first modification example may refer to the metadata attached to the output object light data (n-bit int), or when no metadata is attached, may refer to the metadata attached to the object light data (n-bit int) output previously in time series order.

[0101] Here, the metadata to be referred to is determined by the metadata processing unit 12. The metadata processing unit 12 will be described below.

[0102] First, the metadata processing unit 12 determines whether metadata is attached to the object light data (n-bit int).

[0103] When no metadata is attached to the object light data (n-bit int), the metadata processing unit 12 reads the metadata information from the metadata storage unit 13 and inputs it to the object light inverse quantization apparatus 20 and the object light correction apparatus 30.

[0104] When metadata is attached to the object light data (n-bit int), the metadata processing unit 12 reads a value (object_wave_cancel_flag) indicating whether to process the object light data output from the moving image decoding apparatus 10 by the object light inverse quantization apparatus 20 and the object light correction apparatus 30 from such metadata.

[0105] As shown in step S102 of FIG. 4, when the read object_wave_cancel_flag is "1", the object light decoding device 1 outputs the object light data (n-bit int) output from the moving image decoding device 10. When the object_wave_cancel_flag is "0", the object light inverse quantization device 20 and the object light correction device 30 execute processing.

[0106] When such object_wave_cancel_flag is "1", variables other than object_wave_cancel_flag may not be recorded in the metadata.

[0107] Second, the metadata processing unit 12 reads the object_wave_persistence_flag.

[0108] However, the metadata processing unit 12 may read the object_wave_persistence_flag before or after other metadata variables excluding the object_wave_cancel_flag.

[0109] When the object_wave_persistence_flag is "1", in the processing by the object light inverse quantization device 20 and the object light correction device 30, until the object light data (n-bit int) with newly added metadata is output for the currently decoded object light data (n-bit int) and the object light data (n-bit int) output thereafter, the metadata given to the currently decoded object light data (n-bit int) is used.

[0110] On the other hand, when the object_wave_persistence_flag is "0", only the metadata given to the currently decoded object light data (n-bit int) is used.

[0111] When the object_wave_persistence_flag is "1", the metadata information attached to the currently decoded (n-bit int) is stored in the metadata storage unit 13 attached to the metadata processing unit 12 so that it can be referred to when the object light data (n-bit int) to be output later is processed by the object light inverse quantum device 20 and the object light correction device 30.

[0112] Further, the metadata processing unit 12 may input a flag previous_flag indicating whether the output metadata has been referred to from the metadata storage unit 13 to the wavelength correction unit 32 of the object light correction device 30.

[0113] When previous_flag is "1", it is defined that metadata is referred to from the metadata storage unit 13, and when previous_flag is "0", it is defined that the metadata attached to the input object light data (n-bit int) is referred to.

[0114] As shown in step S108 of FIG. 4, when previous_flag is "1", the correction of object_depth using the correction evaluation unit 34 may be omitted, and instead, the value stored in the object distance recording unit 35 may be referred to as the corrected value of object_depth.

[0115] By omitting the process of correcting object_depth in the correction evaluation unit 34, the processing speed of the wavelength correction unit 32 can be improved.

[0116] According to the above-described modification example 1, by quantizing a plurality of object light data configured in time series order under common conditions, compared with the case of quantizing each object light data under individual conditions, the temporal pixel-to-pixel correlation between the object light data is increased, and inter-frame predictive coding can be efficiently performed.

[0117] (Modification Example 2) Hereinafter, the modification example 2 of the above-described first embodiment will be mainly described with respect to the differences from the above-described first embodiment and modification example 1.

[0118] In the above-described first embodiment and Modification Example 1, the decodable object light data (float) is limited to the object light data on the display device surface (display surface) (see, for example, FIG. 7).

[0119] The object light data on the display surface is obtained by performing diffraction calculation on the light wave emitted from the solid object in the 3D space to the display surface. However, from the perspective of moving image encoding, the object light data on the recording surface set near the solid object has higher pixel-to-pixel correlation in the time direction and the space direction and may have better encoding efficiency than the object light data on the display surface.

[0120] Therefore, by encoding and transmitting the object light near the solid object, decoding it on the object light decoder 1 side, and then performing diffraction calculation from the recording surface near the solid object to the display surface, it is possible to efficiently encode and transmit the object light and then reproduce the solid object on the display surface.

[0121] As shown in FIG. 8, the object light decoder 1 according to this Modification Example 2 includes a diffraction calculation device 40 in addition to the configuration of Modification Example 1 including a moving image decoder 10, an object light inverse quantization device 20, and an object light correction device 30.

[0122] In this Modification Example 2, the metadata processing unit 13 of the moving image decoder 10 reads, as part of the metadata, information indicating the relationship between the recording surface and the display surface of the decoded object light data. Note that object_wave_depth, which is one of such information, is a variable indicating the value of the distance between the recording surface and the display surface.

[0123] Here, the display surface is a surface set at the position of the display device provided in the receiving device, and the recording surface is an arbitrary surface in the decoded 3D space.

[0124] The diffraction calculation device 40 is configured to construct the object light data (float) on the display surface by diffraction calculation from the object light data (float) on an arbitrary recording surface in the 3D space where the decoded object light data exists (see step S111 in FIG. 4).

[0125] Here, the diffraction calculation device 40 can perform such diffraction calculations using existing methods such as the angular spectrum method (plane wave expansion), Fresnel diffraction, and Fraunhofer diffraction.

[0126] Since the distance from the recording surface to the display surface is required in the calculation process of the diffraction calculation, the diffraction calculation device 40 refers to object_wave_depth.

[0127] That is, the diffraction calculation device 40 performs diffraction calculations based on the object light data on the recording surface decoded as a moving image and the information (object_wave_depth) indicating the relationship between the recording surface and the display surface, thereby converting the object light data on the recording surface into object light data on the display surface.

[0128] With reference to FIG. 11, an example of the diffraction calculation performed by the diffraction calculation device 40 will be described.

[0129] In FIG. 11(a), it is assumed that the recording object is composed of three-dimensional object elements 1 to 3, and the three-dimensional object elements 1 to 3 are in order away from the display surface.

[0130] Also, the recording surface may be set between the three-dimensional object elements. In the example of FIG. 11(a), it is located between the three-dimensional object element 1 and the three-dimensional object element 2, and the distance between the recording surface and the display surface is given by the above-mentioned object_wave_depth.

[0131] As shown in FIG. 11(b), the diffraction calculation device 40 obtains the object light data on the display surface by performing the above-mentioned diffraction calculation.

[0132] According to the above-mentioned modification example 2, for the purpose of improving the coding efficiency, the bit stream obtained by encoding the object light data other than the display surface is decoded so that the pixel correlation in the time direction and the space direction becomes high, and it becomes possible to display the reproduced image holography.

[0133] The above-described object decoding device 1, moving image decoding device 10, and diffraction calculation device 40 may be realized by a program that causes a computer to execute each function (each process).

Industrial Applicability

[0134] According to the present embodiment, for example, in moving image communication, since an improvement in overall service quality can be realized, it is possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, "Build resilient infrastructure, promote sustainable industrialization, and foster innovation."

Explanation of Signs

[0135] 1... Object decoding device 10... Moving image decoding device 11... Decoding unit 12... Metadata processing unit 13... Metadata storage unit 20... Object optical inverse quantization device 21... Object optical data discrimination unit 22... Inverse quantization mode selection unit 23... Mapping unit 30... Object optical correction device 31... Sampling interval correction unit 32... Wavelength correction unit 33... Correction calculation unit 34... Correction evaluation unit 35... Object distance recording unit 40... Diffraction calculation device

Claims

1. A moving image decoding device, comprising: a decoding unit that decodes object light data converted into a moving image from a bit stream; and a metadata processing unit that reads metadata, wherein the metadata processing unit reads, as part of the metadata, information indicating the relationship between the recording surface and the display surface of the object light data.

2. The moving image decoding device according to claim 1, wherein the information indicating the relationship between the recording surface and the display surface is the distance between the recording surface and the display surface.

3. The moving image decoding device according to claim 1, wherein the display surface is a surface set at the position of a display device provided in a receiving device.

4. The moving image decoding device according to claim 1, wherein the recording surface is an arbitrary surface within a 3D space where the decoded object light data exists.

5. The moving image decoding device according to any one of claims 1 to 4, wherein the decoding unit decodes the object light data using inter-frame prediction.

6. A diffraction calculation device, comprising: a diffraction calculation unit that converts the decoded object light data into object light data on the display surface based on the object light data decoded as a moving image and information indicating the relationship between the recording surface and the display surface of the object light data.

7. The diffraction calculation device according to claim 6, wherein the information indicating the relationship between the recording surface and the display surface is the distance between the recording surface and the display surface.

8. The diffraction calculation device according to claim 6, wherein the display surface is a surface set at the position of a display device provided in a receiving device.

9. The diffraction calculation device according to claim 6, wherein the recording surface is an arbitrary surface within a 3D space where the decoded object light data exists.

10. An object light decoding device comprising a moving image decoding device, a diffraction calculation device, and an object light correction device, wherein the moving image decoding device comprises a decoding unit that decodes object light data converted into a moving image from a bit stream, and a metadata processing unit that reads metadata, and the metadata processing unit reads, as part of the metadata, information indicating the relationship between the recording surface and the display surface of the object light data, The diffraction calculation device converts the decoded object light data into object light data on the display surface based on the object light data decoded as a moving image and information indicating the relationship between the recording surface and the display surface of the object light data. The object light correction device is an object light decoding device characterized by correcting the object light data on the display surface and outputting object light. **Claim 11** The object light decoding device according to claim 10, wherein the information indicating the relationship between the recording surface and the display surface is the distance between the recording surface and the display surface. **Claim 12** The object light decoding device according to claim 10, wherein the display surface is a surface set at the position of a display device provided in the receiving device. **Claim 13** The object light decoding device according to claim 10, wherein the recording surface is an arbitrary surface within the 3D space where the decoded object light data exists. **Claim 14** The object light decoding device according to any one of claims 10 to 13, wherein the decoding unit decodes the object light data using inter-frame prediction. **Claim 15** An object light decoding method, comprising: decoding object light data converted into a moving image from a bitstream; reading, as part of metadata, information indicating the relationship between the recording surface and the display surface of the object light data; converting the decoded object light data into object light data on the display surface based on the decoded object light data and the information indicating the relationship between the recording surface and the display surface of the object light data; and correcting the object light on the display surface and outputting object light. **Claim 16** A program for causing a computer to function as a diffraction calculation device, wherein the diffraction calculation device converts the decoded object light data into object light data on the display surface based on the object light data decoded as a moving image and information indicating the relationship between the recording surface and the display surface of the object light data. **Claim 17** A program for causing a computer to function as an object light decoding device including a moving image decoding device, a diffraction calculation device, and an object light correction device, wherein the moving image decoding device includes a decoding unit that decodes object light data converted into a moving image from a bitstream, and a metadata processing unit that reads metadata, and the metadata processing unit reads, as part of the metadata, information indicating the relationship between the recording surface and the display surface of the object light data. The diffraction calculation device converts the decoded object light data into object light data on the display surface based on the object light data decoded as a moving image and information indicating the relationship between the recording surface and the display surface of the object light data. The object light correction device is a program characterized by correcting the object light data on the display surface to output object light.

Images