Decoding method and electronic device

Abstract

An embodiment of this application provides a decoding method and an electronic device. The method includes: obtaining a bitstream, the bitstream including intermediate data encoded by a second device; decoding the bitstream to obtain intermediate data; and performing data format conversion on the intermediate data to obtain probe data. The probe data corresponds to one or more probes in a three-dimensional scene, and the probe data is for determining a shading effect of an object in the three-dimensional scene in a rendering process. The data format conversion includes a domain conversion. The second device performs data format conversion on the probe data to achieve a lower bit rate under the same rendering effect or a better rendering effect under the same bit rate compared with the prior art. Then, a decoder of the first device can restore the probe data from the intermediate data through data format conversion, and then determine a shading effect of an object in the three-dimensional scene in a rendering process based on the probe data.

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This application claims priority to Chinese Patent Application No. 202210255747.6, entitled “DECODING METHOD AND ELECTRONIC DEVICE”, filed with the State Intellectual Property Office of the People's Republic of China on March 15, 2022, the entire contents of which are incorporated by reference.

[0002] [Technical field] TECHNICAL FIELD Embodiments of the present application relate to the field of encoding / decoding, and in particular to a decoding method and electronic device. [Background technology]

[0003] As people gradually impose higher requirements on the quality of rendered images, the method of simulating the shading effect in the rendering process has gradually shifted from simulating the shading effect of direct lighting (i.e. simulating the shading effect of a single reflected ray) to simulating the shading effect of indirect lighting (i.e. simulating the shading effect of a multiple reflected ray) to make the image clearer. Probe is a method of simulating the shading effect of direct lighting.

[0004] Currently, in a collaborative scene between a device and a cloud, the cloud generates probe data, compresses the probe data, and sends the compressed probe data to the device side. After receiving the bitstream, the device side decompresses the bitstream to obtain the probe data, and then in the rendering process, calculates the indirect shading effect generated by the light reflected by the object in the 3D (3-dimension) scene based on the probe data obtained through decoding. Summary of the Invention

[0005] This application provides a decoding method and an electronic device. Compared with the prior art, the method, which cooperates with a corresponding encoding method, can reduce the bit rate under the same rendering effect, or improve the rendering effect under the same bit rate.

[0006] According to a first aspect, the application provides a decoding method. The decoding method includes the steps of obtaining a bitstream, the bitstream including intermediate data encoded by a second device, decoding the bitstream to obtain intermediate data, and performing data format conversion on the intermediate data to obtain probe data. The probe data corresponds to one or more probes in a three-dimensional scene, and the probe data is for determining a shading effect of an object in the three-dimensional scene in a rendering process. The object in the three-dimensional scene corresponds to a three-dimensional model in the three-dimensional scene, and the three-dimensional model may include a model of an object or a model of a person, and the data format conversion includes a domain conversion. The second device performs data format conversion on the probe data to achieve a lower bit rate under the same rendering effect or a better rendering effect under the same bit rate compared with the prior art. Thus, the probe data is converted into a more compact representation format for compression, or the number of bits occupied by data of higher importance is increased in the bitstream in the rendering process for compression. In this case, the decoder of the first device can restore the probe data from the intermediate data through data format conversion, and then determine the shading effect of the object in the three-dimensional scene in the rendering process based on the probe data. Therefore, compared with the prior art, when the rendering effect at the decoder side is the same, the rendering delay at the decoder side in the method in this application is shorter, or when the rendering delay at the decoder side is the same, the rendering effect at the decoder side in the method in this application is better. The better rendering effect may mean that the image of the rendered picture enjoys more accurate lighting color, clearer brightness, less light leakage, etc.

[0007] For example, the decoding method in this application may be applied to N (N is an integer greater than 1)-end collaborative rendering scenes, such as cloud games, cloud exhibitions, interior decoration, clothing design and architectural design, which are not limited in this application. The second device may be a server or a terminal, and the first device may be a terminal.

[0008] For example, the decoding method in this application is applied to a device-cloud collaborative rendering scene, where the second device is a server and the first device is a terminal, such as a terminal device, for example, a personal computer, a mobile phone, or a VR (Virtual Reality) wearable device.

[0009] For example, domain transformation may be converting the representation of data from one domain to another. Domains may be classified from different perspectives based on requirements.

[0010] For example, in terms of whether or not normalization is performed, domains may be classified into normalized domains and non-normalized domains.

[0011] In terms of color space, the domain may be classified into the RGB domain, the YUV domain, the XYZ domain and the Lab domain.

[0012] In terms of numerical relationships, the domains may be classified into linear and non-linear domains, which may be, for example, the exponential domain, the PQ (Perceptual Quantization) domain or the HLG (hybrid log-gamma) domain.

[0013] In terms of numerical representation, the domain may be classified into a picture domain and a transform domain. For example, the picture domain may be a domain represented by a picture. For example, the transform domain may be a domain represented by a transform basis function and a corresponding transform coefficient. For data Y(t) in the transform domain, x transform bases e_1(t) to e_x(t) may be used to approximate the data Y(t), so that the data Y(t) is approximately equal to the sum of the x transform bases multiplied by the corresponding transform coefficients. The transform bases include, but are not limited to, spherical harmonic bases, spherical wavelet bases, eigenvectors, etc. This is not limited in this application.

[0014] For example, RGB domain, YUV domain, XYZ domain, and Lab domain refer to the CIE 1931 RGB color space, the YUV color space (including variants such as YCbCr, YPbPr, and YCoCg), the CIE 1931 XYZ color space, and the CIELAB color space, respectively.

[0015] For example, any video decoding scheme may be used to decode the bitstream to obtain intermediate data corresponding to the probe data. For example, HEVC (High Efficiency Video Coding) decoding may be performed on the bitstream to obtain the intermediate data. In another example, AVC (Advanced Video Coding) decoding may be performed on the bitstream to obtain the intermediate data. In another example, VVC (Versatile Video Coding) decoding may be performed on the bitstream to obtain the intermediate data. In another example, entropy decoding may be performed on the bitstream to obtain the intermediate data. In another example, the intermediate data may be encoded with other video decoding schemes, such as AVS (Audio Video coding Standard), to obtain the corresponding bitstream. This is not limited in this application.

[0016] According to a first aspect, the step of performing data format conversion on the intermediate data to obtain the probe data includes a step of performing a first operation on the intermediate data to obtain transformed data, and a step of performing a second operation on the transformed data to obtain the probe data, where when the first operation is a domain transformation, the second operation includes inverse quantization and / or rearrangement in a first manner, and when the second operation is a domain transformation, the first operation includes inverse quantization and / or rearrangement in a first manner. The second device performs data format conversion on the probe data in a manner that combines the domain transformation and the quantization and / or rearrangement in the first manner to further reduce the bitstream. In this case, the decoder of the first device may also restore the probe data from the intermediate data in a manner that combines the domain transformation and the inverse quantization and / or rearrangement in the first manner.

[0017] When a decoder of a first device uses a scheme combining domain transformation and inverse quantization and / or rearrangement in the first manner, and an encoder of a second device uses a scheme combining domain transformation and quantization and / or rearrangement in the first manner, the order and number of times of execution of the domain transformation, quantization and rearrangement in the second device correspond to the order and number of times of execution of the domain transformation, inverse quantization and rearrangement in the first device, it should be understood that the order of execution of the domain transformation, inverse quantization and rearrangement in the first manner is not limited, and the number of times of execution of any one of the domain transformation, inverse quantization and rearrangement is not limited in this application.

[0018] For example, the inverse quantization may be performed based on a quantization parameter, which may correspond to a quantization parameter used by an encoder of the second device to quantize the probe data.

[0019] For example, the conversion data may include at least one of conversion data 1 to conversion data 17 in the following embodiments.

[0020] According to the first aspect or any one of the implementation manners of the first aspect, the step of performing data format conversion on the intermediate data to obtain the probe data further includes a step of performing a third process on the transformed data obtained through the second process after the second process is performed on the transformed data and before the probe data is obtained, the third process including at least one of the following: domain conversion, inverse quantization, or rearrangement in the first manner. The second device performs data format conversion on the probe data in a manner that combines the domain conversion and the quantization and / or the rearrangement in the first manner to further reduce the bitstream. In this case, the decoder of the first device may also restore the probe data from the intermediate data in a manner that combines the domain conversion and the inverse quantization and / or the rearrangement in the first manner.

[0021] According to the first aspect or any one of the implementations of the first aspect, the intermediate data is data on a YUV plane, and before the step of performing the first processing on the intermediate data, the method further includes a step of performing a second type of rearrangement on the intermediate data, the second type of rearrangement including extracting data from the YUV plane. The second device uses a combination of different rearrangement schemes to make the arrangement of the probe data more compact, thereby further reducing the bitstream. In this case, the decoder of the first device may also use a combination of different rearrangement schemes to restore the probe data from the intermediate data.

[0022] It should be noted that when the intermediate data is not data on a YUV plane, the step of performing the second type of rearrangement on the intermediate data may not be performed.

[0023] According to either the first aspect or any one of the implementation manners of the first aspect, the rearrangement in the second manner further includes arranging the data extracted from the YUV plane into a two-dimensional picture.

[0024] It should be noted that when the data extracted from the YUV plane are the transform base and transform coefficients, the rearrangement in the second manner may not include arranging the data extracted from the YUV plane into a two-dimensional picture.

[0025] According to the first aspect or any one of the implementation methods of the first aspect, when the intermediate data is intermediate data corresponding to lighting data, the rearrangement of the first method includes at least one of the following: adding a channel to the intermediate data corresponding to the lighting data, converting the intermediate data corresponding to the lighting data to a data storage format of the first device, or performing a dimensional conversion on the intermediate data corresponding to the lighting data.

[0026] It should be noted that the "intermediate data corresponding to the illumination data" in "converting intermediate data corresponding to the illumination data into a data storage format of the first device" may be original intermediate data in the bitstream corresponding to the illumination data, or intermediate data corresponding to illumination data obtained through domain transformation, or intermediate data corresponding to dequantized illumination data, or intermediate data corresponding to illumination data to which channels have been added, or intermediate data corresponding to illumination data obtained through dimensional transformation.

[0027] It should be noted that the "intermediate data corresponding to the illumination data" in "performing a dimensional transformation on intermediate data corresponding to the illumination data" may be original intermediate data in the bitstream corresponding to the illumination data, or intermediate data corresponding to illumination data obtained through domain transformation, or intermediate data corresponding to dequantized illumination data, or intermediate data corresponding to illumination data to which a channel has been added, or intermediate data corresponding to illumination data whose data format has been converted to the data storage format of the first device.

[0028] For example, the intermediate data may be a matrix. For example, the dimension transformation may be to change the size of the matrix. For example, the intermediate data is a matrix having a size of 50*60*100, and the intermediate data may be transformed into a matrix having a size of 100*1000*3 through the dimension transformation. In another example, the intermediate data is a matrix having a size of 190*190*3, and the intermediate data may be transformed into a matrix having a size of 150*240*3 through the dimension transformation, and redundant invalid data is discarded.

[0029] According to the first aspect or any one of the implementation methods of the first aspect, when the intermediate data is intermediate data corresponding to visibility data and the intermediate data corresponding to the visibility data includes a group of multiple channels, the relocation of the first method includes at least one of the following: combining the group of multiple channels, converting the intermediate data corresponding to the visibility data into a data storage format of the first device, or performing a dimensional transformation on the intermediate data corresponding to the visibility data.

[0030] It should be noted that the "intermediate data corresponding to the visibility data" in "converting intermediate data corresponding to the visibility data into a data storage format of the first device" may be original intermediate data in the bitstream corresponding to the visibility data, or intermediate data corresponding to visibility data obtained through domain transformation, or intermediate data corresponding to dequantized visibility data, or intermediate data corresponding to visibility data to which channels have been added, or intermediate data corresponding to visibility data obtained through dimensional transformation.

[0031] It should be noted that the "intermediate data corresponding to the visibility data" in "performing a dimensional transformation on intermediate data corresponding to the visibility data" may be original intermediate data in the bitstream corresponding to the visibility data, or intermediate data corresponding to visibility data obtained through domain transformation, or intermediate data corresponding to dequantized visibility data, or intermediate data corresponding to visibility data to which channels have been added, or intermediate data corresponding to visibility data whose data format has been converted to the data storage format of the first device.

[0032] It should be noted that combining multiple channel groups means combining multiple channel groups into one channel group, where before the multiple channel groups are combined, each channel group contains at least one channel, and the number of channels in one channel group obtained by combining the multiple channel groups is equal to the total number of channels in the multiple channel groups before combining.

[0033] According to the first aspect or any one of the implementations of the first aspect, the step of performing data format conversion on the intermediate data to obtain the probe data includes the step of performing data format conversion on the intermediate data to obtain the probe data based on the first attribute data obtained by decoding the bitstream. Thus, the decoder of the first device may perform data format conversion on the intermediate data based on the information parsed from the bitstream.

[0034] For example, the decoder of the first device may alternatively acquire information about the data format conversion performed by the encoder of the second device in the encoding process in advance, and after the intermediate data is acquired from the bitstream through decoding, the data format conversion may be performed on the intermediate data based on the information about the data format conversion performed by the encoder of the second device in the encoding process acquired in advance.

[0035] According to the first aspect or any one of the implementations of the first aspect, the first attribute data includes at least one of the following: a quantization parameter, a domain transformation parameter, and a rearrangement parameter.

[0036] For example, the domain transformation parameters may include normalization parameters, exponential transformation parameters, PQ transformation parameters, HLG transformation parameters, and color space transformation parameters, which are not limited in this application.

[0037] According to the first aspect or any one of the implementations of the first aspect, the domain conversion includes at least one of the following: conversion from a normalized domain to a non-normalized domain, conversion from a non-linear domain to a linear domain, conversion from a YUV domain to an RGB domain, conversion from an XYZ domain to an RGB domain, and conversion from a Lab domain to an RGB domain.

[0038] According to the first aspect or any one of the implementation manners of the first aspect, the bitstream further includes attribute data of the probe encoded by the second device, the attribute data including first attribute data for data format conversion and / or second attribute data used in the rendering process.

[0039] According to the first aspect or any one of the implementation methods of the first aspect, when the intermediate data includes intermediate data corresponding to lighting data and intermediate data corresponding to visibility data, the bitstream further includes bitstream structure information, where the bitstream structure information includes a position of the intermediate data corresponding to the lighting data and / or a position of the intermediate data corresponding to the visibility data.

[0040] For example, the bitstream structure information may include, but is not limited to, the number of probes, the location, length and data format of the intermediate data corresponding to the illumination data, the order of execution of the data format conversion types corresponding to the illumination data, the location, length and data format of the intermediate data corresponding to the visibility data, the order of execution of the data format conversion types corresponding to the visibility data, the location, length and data format of the first attribute data, and the location, length and data format of the second attribute data, which is not limited in this application. The data format conversion type may include various data format conversion types used by the second device during encoding, such as encoding side operation types corresponding to various first-scheme rearrangement types (e.g., discarding data of some channels, channel splitting, precision conversion, and dimension conversion), various normalization types (e.g., maximum normalization, fixed parameter normalization, min-max normalization, and z-score normalization), various domain conversion types (e.g., linear domain to nonlinear domain conversion, RGB domain to YUV domain conversion, RGB domain to XYZ domain conversion, RGB domain to Lab domain conversion, and picture domain to transform domain conversion), and various quantization types (e.g., uniform quantization and non-uniform quantization). The second-scheme rearrangement type includes a second rearrangement type used by the second device during encoding, such as padding to the YUV plane. This is not limited in this application.

[0041] According to the first aspect or any one of the implementation methods of the first aspect, when the intermediate data includes intermediate data corresponding to illumination data and intermediate data corresponding to visibility data, the probe data includes illumination data and / or visibility data.

[0042] For example, the illumination data may be for describing the emanating illumination of objects around the probe.

[0043] For example, the visibility data may be for describing the distribution of distances between the probe and objects around the probe (which may also be referred to as a depth distribution), and may include, but is not limited to, distance data, the variance of the distance data, the square of the distance data, etc. This is not a limitation of this application.

[0044] According to a second aspect, the present application provides a first electronic device, the electronic device comprising: a bitstream acquisition module configured to acquire a bitstream, the bitstream including intermediate data encoded by the second device; a decoding module configured to decode the bitstream to obtain intermediate data; a data format conversion module configured to perform a data format conversion on the intermediate data to obtain probe data, the probe data corresponding to one or more probes in the three-dimensional scene, the probe data being for determining a shading effect of an object in the three-dimensional scene in a rendering process, and the data format conversion including a domain conversion; Includes.

[0045] According to a second aspect, the data format conversion module includes: a domain transformation module configured to perform a first operation on the intermediate data to obtain transformed data, the first operation being domain transformation, or a domain transformation module configured to perform a second operation on the transformed data to obtain probe data, the second operation being domain transformation; an inverse quantization module configured to perform a first operation on the intermediate data to obtain transformed data, the first operation being inverse quantization, or an inverse quantization module configured to perform a second operation on the transformed data to obtain probe data, the second operation being inverse quantization; a relocation module configured to perform a first operation on the intermediate data to obtain transformed data, the first operation being a relocation of a first type; or a relocation module configured to perform a second operation on the transformed data to obtain probe data, the second operation being a relocation of the first type; Includes.

[0046] According to the second aspect or any one of the implementation manners of the second aspect, the domain transformation module is further configured to perform a third processing on the transformed data obtained through the second processing after the second processing is performed on the transformed data and before the probe data is obtained, the third processing being a domain transformation.

[0047] The inverse quantization module is further configured to perform a third operation on the transformed data obtained through the second operation after the second operation is performed on the transformed data and before the probe data is obtained, the third operation being inverse quantization.

[0048] The rearrangement module is further configured to perform a third process on the transformation data acquired through the second process after the second process is performed on the transformation data and before the probe data is acquired, the third process being a rearrangement of the first type.

[0049] According to the second aspect or any one of the implementation manners of the second aspect, the intermediate data is data on a YUV plane, and the rearrangement module is further configured to perform a second type of rearrangement on the intermediate data before the first processing is performed on the intermediate data, the second type of rearrangement including extracting data from the YUV plane.

[0050] According to the second aspect or any one of the implementation manners of the second aspect, the rearrangement in the second manner further comprises arranging the data extracted from the YUV plane into a two-dimensional picture.

[0051] According to the second aspect or any one of the implementation methods of the second aspect, when the intermediate data is intermediate data corresponding to lighting data, the rearrangement of the first method includes at least one of the following: adding a channel to the intermediate data corresponding to the lighting data, converting the intermediate data corresponding to the lighting data to a data storage format of the first device, or performing a dimensional conversion on the intermediate data corresponding to the lighting data.

[0052] According to the second aspect or any one of the implementation methods of the second aspect, when the intermediate data is intermediate data corresponding to visibility data and the intermediate data corresponding to the visibility data includes a group of multiple channels, the rearrangement of the first method includes at least one of the following: combining the group of multiple channels, converting the intermediate data corresponding to the visibility data into a data storage format of the first device, or performing a dimensional transformation on the intermediate data corresponding to the visibility data.

[0053] According to the second aspect or any one of the implementation manners of the second aspect, the data format conversion module is specifically configured to perform data format conversion on the intermediate data based on first attribute data obtained by decoding the bitstream to obtain probe data.

[0054] According to the second aspect or any one of the implementations of the second aspect, the first attribute data includes at least one of the following: a quantization parameter, a domain transformation parameter, and a rearrangement parameter.

[0055] According to the second aspect or any one of the implementations of the second aspect, the domain conversion includes at least one of the following: conversion from a normalized domain to a non-normalized domain, conversion from a non-linear domain to a linear domain, conversion from a YUV domain to an RGB domain, conversion from an XYZ domain to an RGB domain, and conversion from a Lab domain to an RGB domain.

[0056] According to the second aspect or any one of the implementation manners of the second aspect, the bitstream further includes attribute data of the probe encoded by the second device, the attribute data including first attribute data for data format conversion and / or second attribute data used in the rendering process.

[0057] According to the second aspect or any one of the implementation methods of the second aspect, when the intermediate data includes intermediate data corresponding to lighting data and intermediate data corresponding to visibility data, the bitstream further includes bitstream structure information, where the bitstream structure information includes a position of the intermediate data corresponding to the lighting data and / or a position of the intermediate data corresponding to the visibility data.

[0058] According to the second aspect or any one of the implementation methods of the second aspect, when the intermediate data includes intermediate data corresponding to illumination data and intermediate data corresponding to visibility data, the probe data includes illumination data and / or visibility data.

[0059] The second aspect and any one of the implementation methods of the second aspect correspond to the first aspect and any one of the implementation methods of the first aspect, respectively. For the technical effects corresponding to the second aspect and any one of the implementation methods of the second aspect, refer to the technical effects corresponding to the first aspect and any one of the implementation methods of the first aspect. Details are not described again here.

[0060] According to a third aspect, an embodiment of the present application provides an electronic device including a memory and a processor, the memory coupled to the processor, the memory storing program instructions which, when executed by the processor, enable the electronic device to perform the decoding method of the first aspect or any one of the possible implementations of the first aspect.

[0061] The third aspect and any one of the implementation methods of the third aspect correspond to the first aspect and any one of the implementation methods of the first aspect, respectively. For the technical effects corresponding to the third aspect and any one of the implementation methods of the third aspect, refer to the technical effects corresponding to the first aspect and any one of the implementation methods of the first aspect. Details are not described again here.

[0062] According to a fourth aspect, an embodiment of the present application provides a chip including one or more interface circuits and one or more processors. The interface circuits are configured to receive signals from a memory of an electronic device and transmit the signals to the processor, the signals including computer instructions. When the processor executes the computer instructions, the electronic device is enabled to perform the decoding method of the first aspect or any one of the possible implementations of the first aspect.

[0063] The fourth aspect and any one of the implementation methods of the fourth aspect correspond to the first aspect and any one of the implementation methods of the first aspect, respectively. For the technical effects corresponding to the fourth aspect and any one of the implementation methods of the fourth aspect, refer to the technical effects corresponding to the first aspect and any one of the implementation methods of the first aspect. Details are not described again here.

[0064] According to a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium storing a computer program, which, when executed on a computer or processor, causes the computer or processor to execute the decoding method of the first aspect or any one of the possible implementations of the first aspect.

[0065] The fifth aspect and any one of the implementation methods of the fifth aspect correspond to the first aspect and any one of the implementation methods of the first aspect, respectively. For the technical effects corresponding to the fifth aspect and any one of the implementation methods of the fifth aspect, refer to the technical effects corresponding to the first aspect and any one of the implementation methods of the first aspect. Details are not described again here.

[0066] According to a sixth aspect, an embodiment of the present application provides a computer program product, comprising a software program, which when executed by a computer or a processor, causes the computer or the processor to perform the decoding method of the first aspect or any one of the possible implementations of the first aspect.

[0067] The sixth aspect and any one of the implementation methods of the sixth aspect correspond to the first aspect and any one of the implementation methods of the first aspect, respectively. For the technical effects corresponding to the sixth aspect and any one of the implementation methods of the sixth aspect, refer to the technical effects corresponding to the first aspect and any one of the implementation methods of the first aspect. Details are not described again here.

[0068] According to a seventh aspect, the present application provides a computer-readable storage medium for storing a bitstream. When the bitstream is decoded by using the decoding method in any one of the first aspect and the implementation manner of the first aspect, probe data may be obtained.

[0069] The seventh aspect and any one of the implementation methods of the seventh aspect correspond to the first aspect and any one of the implementation methods of the first aspect, respectively. For the technical effects corresponding to the seventh aspect and any one of the implementation methods of the seventh aspect, refer to the technical effects corresponding to the first aspect and any one of the implementation methods of the first aspect. Details are not described again here.

[0070] According to an eighth aspect, the application provides a bitstream. When the bitstream is decoded by using the decoding method in the first aspect and any of the implementation manners of the first aspect, probe data may be obtained. Optionally, the bitstream may be stored in a computer-readable storage medium or transmitted in the form of an electromagnetic signal.

[0071] The eighth aspect and any one of the implementation methods of the eighth aspect correspond to the first aspect and any one of the implementation methods of the first aspect, respectively. For the technical effects corresponding to the eighth aspect and any one of the implementation methods of the eighth aspect, refer to the technical effects corresponding to the first aspect and any one of the implementation methods of the first aspect. Details are not described again here. [Brief description of the drawings]

[0072] [Figure 1a] FIG. 1 is a diagram of an example system framework. [Figure 1b] FIG. 1 is an example of a probe distribution in a three-dimensional scene. [Figure 2a] FIG. 1 is a diagram of an example of a decoding framework. [Figure 2b] FIG. 2 is a diagram of an example of the structure of a data format conversion module. [Figure 3a] FIG. 13 is a diagram of an example of a decoding procedure. [Figure 3b] FIG. 13 is a diagram of an example of a decoding procedure. [Figure 4a] FIG. 13 is a diagram of an example of a decoding procedure. [Figure 4b] FIG. 13 is a diagram of an example of a decoding procedure. [Figure 4c(1)] FIG. 13 is a diagram of an example of a decoding procedure. [Figure 4c(2)] FIG. 13 is a diagram of an example of a decoding procedure. [Figure 4c(3)] FIG. 13 is a diagram of an example of a decoding procedure. [Figure 4c(4)] FIG. 13 is a diagram of an example of a decoding procedure. [Figure 4c(5)] FIG. 13 is a diagram of an example of a decoding procedure. [Figure 4c(6)] FIG. 13 is a diagram of an example of a decoding procedure. [Figure 4d] FIG. 2 is a diagram of an example of a bitstream structure. [Diagram 5] FIG. 13 is a diagram of an example of a decoding procedure. [Figure 6] FIG. 13 is a diagram of an example of a decoding procedure. [Figure 7] FIG. 13 is a diagram of an example of a decoding procedure. [Figure 8] FIG. 13 is a diagram of an example of a decoding procedure. [Figure 9] FIG. 13 is a diagram of an example of a decoding procedure. [Figure 10] FIG. 1 is a diagram of an example of an electronic device. [Figure 11a] FIG. 13 is an example of the compression effect. [Figure 11b] FIG. 13 is an example of the compression effect. [Figure 12] FIG. 2 is a diagram of an example of the structure of the device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0073] The following clearly and completely describes the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. It is clear that the described embodiments are only a part, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art according to the embodiments of this application without creative efforts shall fall within the protection scope of this application.

[0074] The term "and / or" in this specification describes only an association relationship for describing related objects and indicates that three relationships may exist. For example, A and / or B may represent the following three cases: only A exists, both A and B exist, and only B exists.

[0075] In the description and claims of the embodiments of this application, the terms "first", "second", etc. are intended to distinguish between different objects, and do not indicate a particular order of the objects. For example, a first target object, a second target object, etc. are intended to distinguish between different target objects, and do not describe a particular order of the target objects.

[0076] In the embodiments of this application, words such as "example" or "for example" are intended to provide an example, illustration, or explanation. Any embodiment or design manner described as "example" or "for example" in the embodiments of this application should not be described as being preferred or having more advantages over other embodiments or design manners. Strictly speaking, the use of words such as "example", "for example", etc. is intended to present the relevant concept in a particular manner.

[0077] In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more. For example, a plurality of processing units means two or more processing units, and a plurality of systems means two or more systems.

[0078] For example, embodiments of this application may be applied to an N-end (ie, N devices) collaborative rendering scene, where N is an integer greater than 1.

[0079] In a possible scene, one device may generate rendering input information (the rendering input information may include one or more of a three-dimensional model (which may also be called a 3D (3-dimension) object model, including a person model and / or an object model), probe data, etc., which is not limited in this application, and an example in which the rendering input information is probe data is used for the description in this application), and then distribute the probe data to the other N-1 devices. After receiving the probe data, the other N-1 devices may determine shading effects of objects (corresponding to the three-dimensional model) in the three-dimensional scene based on the probe data in a rendering process. After the rendering is completed, a rendered picture may be obtained.

[0080] In a possible scene, N1 (where the value of N1 ranges from 2 to N, N1 may be equal to 2 or N, and N1 is an integer) devices may jointly generate probe data, and each of the N1 devices generates a portion of the probe data. Then, each of the N1 devices distributes the portion of the probe data generated by the device to the other N-1 devices. After receiving the probe data, the N1 devices may determine, in a rendering process, shading effects of objects in the three-dimensional scene based on the received probe data and the portion of the probe data generated by the N1 devices. After the rendering is completed, a rendered picture may be obtained. After receiving the probe data, the other N-N1 devices may determine, in a rendering process, shading effects of objects in the three-dimensional scene based on the received probe data. After the rendering is completed, a rendered picture may be obtained.

[0081] For ease of explanation, a device for rendering that determines the shading effect of an object in a three-dimensional scene based on the probe data in the rendering process may be referred to as a first device, and a device that generates the probe data in the N-end collaborative rendering scene may be referred to as a second device. Either device may be a first device or a second device. This is not limited in this application. The second device may be a server or a terminal, and the first device may be a terminal.

[0082] FIG. 1a is a diagram of an example of a system framework. In the embodiment in FIG. 1a, the second device is a computing center server located on a cloud, and the first device is a client. FIG. 1a is a diagram of an example of a framework of a collaborative rendering system between a device and a cloud.

[0083] See Fig. 1a. For example, the device-cloud collaborative rendering system may include a computing center server, an edge server, and a client. There may be n edge servers (n is an integer greater than 1), and there may be k1+k2+...+kn clients, where k1, k2,..., and kn are all positive integers. The computing center server is connected to n edge servers, and each edge server is connected to at least one client. As shown in Fig. 1a, edge server 1 is connected to k1 clients, i.e., client 11, client 12,..., and client k1, edge server 2 is connected to k2 clients, i.e., client 21, client 22,..., and client k2,..., edge server n is connected to kn clients, i.e., client n1, client n2,..., and client kn.

[0084] For example, a client may access the network through a wireless access point, such as a base station or a Wi-Fi access point, and communicate with an edge server over the network, or the client may communicate with the edge server through a wired connection. Alternatively, an edge server may access the network through a wireless access point, such as a base station or a Wi-Fi access point, and communicate with a computing center server over the network, or the edge server may communicate with a computing center server through a wired connection.

[0085] For example, the computing center server may be a single server, a server cluster including multiple servers, or other distributed system, which is not a limitation of this application.

[0086] For example, the number n of edge servers is not limited in this application and may be specifically set based on the actual application scenario, which is not limited in this application. In a possible scenario, the client may be directly connected to the computing center server, and some clients may be directly connected to the computing center server instead of being connected to the edge server. In another possible scenario, all clients are directly connected to the computing center server instead of being connected to the edge server.

[0087] For example, the number of clients connected to each edge server is not limited in this application and may be specifically set based on the actual application scenario. Furthermore, the number of clients connected to the edge servers may be the same or different (i.e., k1, k2, ... and kn may be equal or different), and may be specifically set based on the actual application scenario, which is not limited in this application.

[0088] For example, the client may include, but is not limited to, a mobile phone, a personal computer (PC), a notebook computer, a tablet computer, a smart TV, a mobile Internet device (MID), a wearable device (e.g., a smart watch, smart glasses, or a smart helmet), a virtual reality (VR) device, an augmented reality (AR) device, a wireless electronic device in industrial control, a wireless electronic device in self driving, a wireless electronic device in remote medical surgery, a wireless electronic device in a smart grid, a wireless electronic device in transportation safety, a wireless electronic device in a smart city, a wireless electronic device in a smart home, etc. The specific form of the client is not particularly limited in the following embodiments.

[0089] It should be understood that the framework of the device-cloud collaborative rendering system shown in FIG. 1a is merely an example of the framework of the device-cloud collaborative rendering system in this application. In the device-cloud collaborative rendering system in this application, the computing center server and the edge server may be the same server. Alternatively, in the device-cloud collaborative rendering system in this application, the edge server is not included, and the computing center server is connected to each client. This is not limited in this application. This application is described by using the framework of the device-cloud collaborative rendering system shown in FIG. 1a as an example.

[0090] For example, the computing center service may be configured to generate probe data, such as probe data shared by all clients and personalized probe data for each client.

[0091] For example, the edge server may be configured to distribute probe data and generate personalized probe data for clients connected to the edge server. For example, when the computing center server generates probe data shared by all clients and personalized probe data for each client, the edge server may distribute the probe data generated by the computing center server. In another example, when the computing center server generates only probe data shared by all clients, the edge server may generate personalized probe data for clients connected to the edge server and distribute the probe data generated by the computing center server and the probe data generated by the edge server.

[0092] For example, the client may be configured to perform rendering and display the rendered picture. During the rendering process, shading effects of objects in the three-dimensional scene may be determined based on the probe data.

[0093] For example, N-end collaborative rendering scenes such as cloud gaming, cloud exhibitions, interior decoration, clothing design, and architectural design may be realized by using the device-cloud collaborative rendering system framework shown in FIG. 1a.

[0094] For example, in a cloud game scene, after receiving a view switching command sent by the client 11, the computing center server may generate probe data of the game scene corresponding to the target view, and send the probe data to the edge server 1. The edge server 1 sends the probe data to the client 11. After receiving the probe data, the client 11 may perform rendering and determine the shading effect of the object in the game scene corresponding to the target view. After the rendering is completed, a picture of the game scene corresponding to the target view may be obtained for display.

[0095] For example, in an interior decoration scene, after receiving a furniture addition instruction sent by the client 21, the computing center server may generate probe data corresponding to the living room scene to which the target furniture is added, and send the probe data to the edge server 2. The edge server 2 sends the probe data to the client 21. After receiving the probe data, the client 21 may perform rendering and determine shading effects of objects in the living room scene to which the target furniture is added. After the rendering is completed, a picture of the living room to which the target furniture is added may be acquired for display.

[0096] To facilitate the following description, the following describes a process in which the computing center server generates probe data, and the client determines the shading effect of an object in a three-dimensional scene based on the probe data in a rendering process.

[0097] The process by which the computing center server generates the probe data is as follows.

[0098] For example, the rendering process of the computing center server may be as follows: load a three-dimensional model (which may include a person model or an object model) into a three-dimensional scene (which may be referred to as a 3D scene) (thus, the three-dimensional model may be converted into an object in the three-dimensional scene), and render the object in the three-dimensional scene to obtain a current frame (i.e., a rendered picture). In the process of rendering the object in the three-dimensional scene, multiple probes may be placed in the three-dimensional scene, and the probes are used to detect the surrounding environment and obtain probe data. Then, the shading effect of the object in the three-dimensional scene is determined based on the probe data. This can simulate the shading effect of the object in the three-dimensional scene after the light ray in the three-dimensional scene is reflected multiple times.

[0099] Figure 1b is a diagram of an example of a probe distribution in a 3D scene. Each small ball in Figure 1b represents a probe. In the embodiment in Figure 1b, the probes are Dynamic Diffuse Global Illumination (DDGI) probes.

[0100] See Fig. 1b. For example, the position where each probe is placed in the three-dimensional scene and the positional relationship between each probe and other probes may be set based on requirements. This is not limited in this application. For example, in Fig. 1b, the distances between each probe and the six probes in six directions around the probe (upper right direction, lower right direction, right front direction, right rear direction, right left direction, and right right direction) are equal. Furthermore, the number of probes placed in the three-dimensional scene may also be set based on requirements. This is also not limited in this application.

[0101] After multiple probes are placed in a three-dimensional scene, corresponding attribute data (the attribute data is used in the rendering process) may be configured for each probe based on scene requirements. The attribute data includes, but is not limited to, the type of probe (e.g., reflective probe or DDGI probe), the activation identifier of the probe, the position of the probe, the position offset of the probe (e.g., after the probes are placed in a preset manner, the initial position of the probe may be obtained, and the positions of some probes may be adjusted to obtain a better shading effect, and for a probe, the offset between the adjusted position and the initial position may be called the position offset of the probe, for example, after the probes are placed in the manner shown in FIG. 1b, the distance between each probe and the six probes around it is equal, and when the position of the probe is adjusted, the distance between the probe and the six probes around it is not equal), etc. This is not limited in this application.

[0102] For example, after multiple probes are placed in a three-dimensional scene, each probe may detect the surrounding environment around the probe, i.e., detect features of surrounding objects around the probe in the three-dimensional scene, and record the features as the environment data of the probe. The environment data may include lighting data, color data, visibility data, normal direction, texture coordinates, material information, etc. The lighting data may be for describing the emissive lighting of objects around the probe. The visibility data may be for describing the distribution of distances between the objects around the probe and the probe (also called depth distribution), and may include, but is not limited to, distance data, variance of distance data, square of distance data, etc. This is not limited in this application. In the following, an example is provided for illustration by using an example of generating lighting data and visibility data.

[0103] For example, a DDGI algorithm may be used to generate lighting data and visibility data corresponding to each probe. In the following, a probe in a current frame is used as an example to describe the process of generating lighting data and visibility data of a probe. First, several rays emitted from the probe are sampled, and a first intersection point of the several rays with an object in a three-dimensional scene is calculated. Then, a distance between the probe and a first intersection point of each of the several rays of the probe with an object in a three-dimensional scene is calculated to obtain initial distance data. Illumination at the first intersection point of the several rays with an object in a three-dimensional scene is calculated to obtain initial illumination data. Then, the discrete initial distance data may be converted into continuous spherical data. Specifically, a filtering process may be performed on the initial distance data according to a cos^k kernel function (k is a positive integer) on a sphere to obtain candidate distance data. Furthermore, the discrete initial distance data may be converted into continuous spherical data. Specifically, a filtering process may be performed on the square of the initial distance data according to a cos^k kernel function (k is a positive integer) on a sphere to obtain square of the candidate distance data. Furthermore, the discrete initial illumination data may be converted into continuous spherical data. Specifically, a filtering process may be performed on the initial illumination data according to a cosine kernel function on the sphere to obtain candidate illumination data. Then, a weighting calculation is performed on the candidate distance data of the probe and the distance data of the probe in the previous frame to obtain the distance data of the probe in the current frame. A weighting calculation is performed on the square of the candidate distance data of the probe and the square of the distance data of the probe in the previous frame to obtain the square of the distance data of the probe in the current frame. A weighting calculation is performed on the candidate illumination data of the probe and the illumination data of the probe in the previous frame to obtain the illumination data of the probe in the current frame. In this way, the illumination data and visibility data of all the probes in the current frame can be obtained.

[0104] For example, both the illumination data and the visibility data of each probe may be represented by a two-dimensional picture, or may be represented by spherical harmonics function basis coefficients, or may be represented by spherical wavelet basis coefficients, which is not a limitation of this application.

[0105] It should be noted that the 3D scene is assumed to include M probes (M is a positive integer). M1 probes have any one of lighting data, visibility data and attribute data, M2 probes have any two of lighting data, visibility data and attribute data, M3 probes have lighting data, visibility data and attribute data, and M4 probes have no probe data. M1+M2+M3+M4=M, where M1, M2, M3 and M4 are all integers, and the values ​​of M1, M2, M3 and M4 may be set based on requirements, which is not limited in this application.

[0106] The process by which the client determines the shading effect of an object in a three-dimensional scene based on the probe data in the rendering process is as follows.

[0107] For example, the probe data may be for calculating the shading effect of an object in a three-dimensional scene in a rendering process of a client. Specifically, when each pixel is rendered, the coordinates in the 3D space corresponding to the pixel are obtained, and eight probes around the coordinates are searched. Then, the weight given to the pixel by each probe is calculated based on the visibility data of the probe, that is, whether the probe and the 3D coordinates of the probe are visible to each other is determined based on the distance. If the probe and the 3D coordinates of the probe are not visible to each other, the weight is 0. If the probe and the 3D coordinates of the probe are visible to each other, the weight given by the probe is calculated based on the distance data, the square of the distance data, and the position of the probe. Then, a weighted average is performed on the lighting data of the probe based on the given weight to obtain the shading result of the pixel.

[0108] For example, the probe data may include attribute data and environmental data that is used in the rendering process.

[0109] Because the amount of probe data is large, the computing center server can compress the probe data and send the corresponding bitstream to the client to reduce the bandwidth occupied by the server and the client and the rendering delay of the client. Correspondingly, after receiving the bitstream, the client can decode the bitstream to restore the probe data, and determine the shading effect of the object in the three-dimensional scene according to the probe data in the rendering process.

[0110] FIG. 2a is a diagram of an example decoding framework.

[0111] See Figure 2a. For example, the decoder may include a data format conversion module, a first rearrangement module and a decoding module.

[0112] For example, the data format conversion module may be configured to perform a data format conversion on a portion of the data obtained by decoding the bitstream to obtain environmental data of the probe.

[0113] For example, the first rearrangement module may be configured to rearrange a portion of the data obtained by decoding the bitstream to obtain attribute data of the probe, which may include attribute data for data format conversion (hereinafter referred to as first attribute data) and attribute data used in the rendering process (hereinafter referred to as second attribute data).

[0114] For example, the decoding module is configured to decode a bitstream.

[0115] It should be noted that the steps performed by the data format conversion module and the first rearrangement module are steps in the decoding procedure of the decoder.

[0116] It should be understood that Fig. 2a is merely an example of a decoder in this application. The decoder in this application may have fewer modules than those in Fig. 2a. For example, the decoder includes a data format conversion module and a decoding module. This is not limited in this application. Alternatively, the decoder in this application may have more modules than those shown in Fig. 2a. This is not limited in this application.

[0117] It should be understood that the data format conversion module, the first rearrangement module and the decoding module in Fig. 2a may be independent modules, or any two or more of the modules may be integrated as a whole. This is not limited in this application. Furthermore, the data format conversion module, the first rearrangement module and the decoding module are logical modules, and the decoder may be further divided into other modules, or the modules may have other names. This is also not limited in this application.

[0118] It should be understood that in a possible example, the decoder may only include a decoding module, a data format conversion module and a first rearrangement module, which may be independent of the decoder. This is not limited in this application. This application provides an example for explanation by using the decoder in Figure 2a as an example.

[0119] FIG. 2b is a diagram of an example of the structure of a data format conversion module.

[0120] See Figure 2b. For example, the data format conversion module may include an inverse quantization module, a domain conversion module and a second rearrangement module.

[0121] For example, the inverse quantization module may be for inverse quantization. It should be understood that inverse quantization is the inverse process of quantization.

[0122] For example, the domain transformation module may be used for domain transformation. For example, domain transformation may be to transform the representation of data from one domain to another. Domains may be classified from different perspectives based on requirements.

[0123] For example, in terms of whether or not normalization is performed, domains may be classified into normalized domains and non-normalized domains.

[0124] In terms of color space, the domain may be classified into the RGB domain, the YUV domain, the XYZ domain, the Lab domain, etc.

[0125] In terms of numerical relationships, the domains may be classified into linear and non-linear domains, which may be, for example, the exponential domain, the PQ (Perceptual Quantization) domain or the HLG (hybrid log-gamma) domain.

[0126] In terms of numerical representation, the domain may be classified into a picture domain and a transform domain. For example, the picture domain may be a domain represented by a picture. For example, the transform domain may be a domain represented by a transform basis function and a corresponding transform coefficient. For data Y(t) in the transform domain, x transform bases e_1(t) to e_x(t) may be used to approximate the data Y(t), so that the data Y(t) is approximately equal to the sum of the x transform bases multiplied by the corresponding transform coefficients. The transform bases include, but are not limited to, spherical harmonic bases, spherical wavelet bases, eigenvectors, etc. This is not limited in this application.

[0127] For example, the second relocation module may be configured to relocate the data.

[0128] It should be understood that Fig. 2b is merely an example of the data format conversion module in this application. The data format conversion module in this application may have fewer modules than those in Fig. 2b. For example, the data format conversion module only includes an inverse quantization module and a domain conversion module, or the data format conversion module only includes a domain conversion module and a second rearrangement module. This is not limited in this application. Alternatively, the data format conversion module in this application may have more modules than those in Fig. 2b. This is not limited in this application.

[0129] It should be understood that the inverse quantization module, the domain transformation module and the second rearrangement module in Fig. 2b may be independent modules, or any two or more of the modules may be integrated as a whole. This is not limited in this application. Furthermore, the inverse quantization module, the domain transformation module and the second rearrangement module are logical modules, and the data format conversion module may be further divided into other modules, or the modules may have other names. This is also not limited in this application.

[0130] The following describes the decryption process of the first device by using an example.

[0131] 3a is a diagram of an example of a decoding procedure. The decoding procedure may include the following steps: S301: Obtain a bitstream; S302: Perform decoding; S303: Perform data format conversion.

[0132] S301: Obtain a bitstream, that is, obtain a bitstream, where the bitstream includes intermediate data encoded by a second device.

[0133] For example, in an N-end collaborative rendering scene, the second device may generate probe data of K probes (K is a positive integer, K is less than or equal to M) in the current frame based on the above description, and input the probe data of the K probes to the encoder. In this way, the encoder may obtain the probe data of the K probes. Then, the encoder may perform data format conversion on the probe data of the K probes to obtain intermediate data, encode the intermediate data to obtain a corresponding bitstream, and send the bitstream to the first device. The encoder performs data format conversion on the probe data to convert the probe data into data of a more compact representation format, so that compression is easier. Thus, compared with the prior art, the bitrate can be reduced while ensuring the same rendering effect of the first device. Alternatively, the encoder performs data format conversion on the probe data to increase the number of bits occupied by data of higher importance in the rendering process in the bitstream. Thus, compared with the prior art, the rendering effect of the first device can be made better under the same bitrate.

[0134] For example, after the second device transmits a bitstream obtained by encoding intermediate data corresponding to the probe data, the first device may receive the bitstream and input the bitstream to a decoder, so that the decoder can obtain the bitstream.

[0135] S302: Perform decoding, ie, decode the bitstream obtained in S301 to obtain intermediate data.

[0136] For example, after obtaining a bitstream, a decoder may decode the bitstream to obtain intermediate data corresponding to the K probe data.

[0137] For example, any video decoding scheme may be used to decode the intermediate data to obtain a bitstream corresponding to the probe data. For example, HEVC (High Efficiency Video Coding) decoding may be performed on the bitstream to obtain the intermediate data. In another example, AVC (Advanced Video Coding) decoding may be performed on the bitstream to obtain the intermediate data. In another example, VVC (Versatile Video Coding) decoding may be performed on the bitstream to obtain the intermediate data. In another example, entropy decoding may be performed on the bitstream to obtain the intermediate data. It should be understood that the decoding scheme of the bitstream is not limited in this application. In another example, the intermediate data may be encoded with other video decoding schemes, such as AVS (Audio Video coding Standard), to obtain the corresponding bitstream. This is not limited in this application.

[0138] S303: Perform data format conversion, that is, perform data format conversion on the intermediate data obtained in S302 to obtain probe data.

[0139] For example, after obtaining the intermediate data, the decoder of the first device may perform data format conversion on the intermediate data to recover the probe data from the intermediate data.

[0140] For example, the data format conversion may be a conversion from one data format to another. It should be understood that the data format conversion of the decoder of the first device is the reverse process of the data format conversion of the encoder of the second device.

[0141] For example, data format conversion may include domain conversion.

[0142] For example, the domain transformation may include at least one of the following: a normalized domain to a non-normalized domain transformation, a non-linear domain to a linear domain transformation, a YUV domain to an RGB domain transformation, an XYZ domain to an RGB domain transformation, and a Lab domain to an RGB domain transformation. It should be understood that this application may further include other types of domain transformation, which is not limited in this application.

[0143] For example, the transformation from the normalized domain to the non-normalized domain is an inverse normalization. When the domain transformation is a transformation from the normalized domain to the non-normalized domain, an inverse normalization may be performed on the intermediate data based on the normalization parameters to obtain probe data of the K probes. The normalization parameters may correspond to normalization parameters used by the encoder of the second device to normalize the probe data, may be obtained in advance by the decoder of the first device, or may be obtained by decoding the bitstream. This is not limited in this application.

[0144] The second device normalizes the probe data and converts the probe data into a more compact representation format for compression to achieve a lower bit rate under the same rendering effect as in the prior art, in which case the decoder of the first device can restore the probe data by performing denormalization on the intermediate data.

[0145] For example, the intermediate data obtained by decoding the bitstream is data in a non-linear domain.

[0146] For example, when the nonlinear domain is an exponential domain, a conversion from the exponential domain to a linear domain may be performed on the intermediate data based on the exponential conversion parameters to obtain the probe data of K probes. In this case, the probe data of K probes belongs to the linear domain. The exponential conversion parameters may be obtained in advance by the decoder of the first device, or may be obtained by decoding the bitstream. This is not limited in this application.

[0147] For example, when the nonlinear domain is a PQ domain, a transformation from the PQ domain to a linear domain may be performed on the intermediate data based on the PQ transformation parameters to obtain the probe data of the K probes. In this case, the probe data of the K probes belongs to the linear domain. The PQ transformation parameters may be obtained in advance by the decoder of the first device, or may be obtained by decoding the bitstream. This is not limited in this application.

[0148] For example, when the nonlinear domain is an HLG domain, a conversion from the HLG domain to a linear domain may be performed on the intermediate data based on the HLG conversion parameters to obtain the probe data of K probes. In this case, the probe data of the K probes belongs to the linear domain. The HLG conversion parameters may be obtained in advance by the decoder of the first device, or may be obtained by decoding the bitstream. This is not limited in this application.

[0149] The second device performs linear to nonlinear domain transformation on the probe data to increase the number of bits occupied by data for compression of higher importance required in the rendering process in the bitstream, thereby achieving better rendering effect under the same bit rate as in the prior art. In this case, the decoder of the first device can restore the probe data by performing nonlinear to linear domain transformation on the intermediate data.

[0150] For example, the intermediate data obtained by decoding the bitstream may be data represented by YUV.

[0151] In a possible manner, when the intermediate data obtained by decoding the bitstream is data represented by YUV 444, a conversion from the YUV domain to the RGB domain may be performed on the intermediate data in the manner of a conversion from the YUV 444 color space to the RGB color space to obtain probe data of the K probes, in which the probe data of the K probes is represented by RGB.

[0152] In a possible manner, when the intermediate data obtained by decoding the bitstream is represented by YUV 420, a conversion from the YUV domain to the RGB domain may be performed on the intermediate data in the manner of a conversion from the YUV 420 color space to the RGB color space to obtain probe data of the K probes, in which the probe data of the K probes is represented by RGB.

[0153] For example, the intermediate data obtained by decoding the bitstream may be data represented by XYZ, and a transformation from the XYZ domain to the RGB domain may be performed on the intermediate data in the manner of a transformation from the XYZ color space to the RGB color space to obtain probe data of K probes, in which the probe data of the K probes is represented by RGB.

[0154] For example, the intermediate data obtained by decoding the bitstream may be data represented by Lab, and a conversion from the Lab domain to the RGB domain may be performed on the intermediate data in the manner of a conversion from the Lab color space to the RGB color space to obtain the probe data of the K probes, in which the probe data of the K probes is represented by RGB.

[0155] The second device performs RGB domain to YUV domain conversion on the probe data of the K probes to remove redundancy in the probe data of each probe, thereby achieving a lower bit rate under the same rendering effect as the prior art, or performs RGB domain to XYZ domain / Lab domain conversion on the probe data of the K probes to adapt the data to the perception of the human eye, thereby achieving a lower bit rate under the same rendering effect as the prior art. In this case, the decoder of the first device may restore the probe data by performing YUV domain / XYZ domain / Lab domain to RGB domain conversion on the intermediate data.

[0156] It should be understood that two or more of the multiple types of domain transformations may be used to transform the data format of the intermediate data. This is not limited in this application. When two or more types of domain transformations are used to realize the data format transformation of the intermediate data, the execution order and execution number of various types of domain transformations are not limited in this application, under the premise that the execution order and execution number correspond to those in the domain transformation process of the decoder of the second device.

[0157] For example, assume that the data format conversion of the intermediate data is realized through denormalization and conversion from the exponential domain to the linear domain. In a possible manner, denormalization may be performed on the intermediate data, and then conversion from the exponential domain to the linear domain is performed on the intermediate data obtained through denormalization to obtain the probe data. In a possible manner, conversion from the exponential domain to the linear domain may be performed on the intermediate data, and then denormalization is performed on the intermediate data in the linear domain to obtain the probe data.

[0158] In another example, assume that the data format conversion of the intermediate data is realized through inverse normalization, conversion from YUV domain to RGB domain, and conversion from exponential domain to linear domain. In a possible manner, inverse normalization may be performed on the intermediate data, and conversion from exponential domain to linear domain is performed on the intermediate data obtained through inverse normalization, and then conversion from YUV domain to RGB domain is performed on the data in the linear domain to obtain the probe data. In a possible manner, inverse normalization may be performed on the intermediate data, and conversion from YUV domain to RGB domain is performed on the intermediate data obtained through inverse normalization, and then conversion from exponential domain to linear domain is performed on the data in the RGB domain to obtain the probe data. In a possible manner, inverse normalization may be performed on the intermediate data based on a first normalization parameter, and inverse normalization may be performed on the intermediate data obtained through inverse normalization based on another normalization parameter for a second time, and conversion from YUV domain to RGB domain is performed on the intermediate data obtained through two inverse normalizations, and then conversion from exponential domain to linear domain is performed on the data in the RGB domain to obtain the probe data.

[0159] It should be noted that the step in S303 is a step in the decoding procedure.

[0160] For example, when the bitstream includes intermediate data corresponding to illumination data encoded by the second device, the decoder of the first device obtains illumination data through decoding. When the bitstream includes intermediate data corresponding to visibility data encoded by the second device, the decoder of the first device obtains visibility data through decoding. When the bitstream includes intermediate data corresponding to illumination data and intermediate data corresponding to visibility data encoded by the second device, the decoder of the first device obtains illumination data and visibility data through decoding. In other words, in the embodiment in Fig. 3a, the probe data obtained by the decoder of the first device through decoding may include illumination data and / or visibility data.

[0161] The second device performs data format conversion on the probe data to achieve a lower bit rate under the same rendering effect or a better rendering effect under the same bit rate compared with the prior art. Thus, the probe data is converted into a more compact representation format for compression, or the number of bits occupied by data of higher importance increases in the bitstream in the rendering process for compression. In this case, the decoder of the first device can restore the probe data from the intermediate data through data format conversion, and then determine the shading effect of the object in the three-dimensional scene in the rendering process based on the probe data.

[0162] FIG. 3b is a diagram of an example of a decoding procedure.

[0163] See Fig. 3b(1). For example, the acquired bitstream includes intermediate data corresponding to illumination data. First, HEVC decoding (or other decoding) may be performed on the bitstream to obtain the intermediate data corresponding to the illumination data. Then, a YUV domain to RGB domain conversion is performed on the intermediate data corresponding to the illumination data to obtain the illumination data.

[0164] See Fig. 3b(2). For example, the obtained bitstream includes intermediate data corresponding to illumination data. First, HEVC decoding (or other decoding) may be performed on the bitstream to obtain intermediate data corresponding to illumination data. Then, a conversion from exponential domain to linear domain is performed on the intermediate data corresponding to illumination data to obtain illumination data.

[0165] See Fig. 3b(3). For example, the obtained bitstream includes intermediate data corresponding to visibility data. First, entropy decoding (or other decoding) may be performed on the bitstream to obtain the intermediate data corresponding to the visibility data. Then, inverse normalization is performed on the visibility data corresponding to the illumination data to obtain the visibility data.

[0166] See Fig. 3b(4). For example, the obtained bitstream includes intermediate data corresponding to illumination data. First, entropy decoding (or other decoding) may be performed on the bitstream to obtain the intermediate data corresponding to the illumination data. Then, conversion from the YUV domain to the RGB domain and denormalization are performed on the intermediate data corresponding to the illumination data to obtain the illumination data.

[0167] See Fig. 3b(5). For example, the obtained bitstream includes intermediate data corresponding to illumination data and intermediate data corresponding to visibility data. First, HEVC decoding (or other decoding) may be performed on the bitstream to obtain intermediate data corresponding to illumination data and intermediate data corresponding to visibility data. Then, exponential domain to linear domain conversion and inverse normalization are performed on the intermediate data corresponding to illumination data and the intermediate data corresponding to visibility data to obtain illumination data and visibility data.

[0168] For example, the data format conversion may further include inverse quantization and / or rearrangement. Thus, the data format conversion may be performed on the probe data in a manner that combines domain transformation and inverse quantization, or in a manner that combines domain transformation and rearrangement, or in a manner that combines domain transformation, inverse quantization and rearrangement. The second device performs data format conversion on the probe data in a manner that combines domain transformation and quantization and / or rearrangement to further reduce the bitstream. In this case, the decoder of the first device may also recover the probe data from the intermediate data.

[0169] When a decoder of a first device uses a scheme that combines domain transformation and inverse quantization and / or rearrangement, and an encoder of a second device uses a scheme that combines domain transformation and quantization and / or rearrangement, the order and number of times of execution of the domain transformation, quantization and rearrangement of the second device correspond to the order and number of times of execution of the domain transformation, inverse quantization and rearrangement of the first device, it should be understood that the order of execution of the domain transformation, inverse quantization and rearrangement is not limited in this application, and the number of times of execution of any one of the domain transformation, inverse quantization and rearrangement is not limited in this application.

[0170] For example, the inverse quantization may be performed based on a quantization parameter, which may correspond to a quantization parameter used by an encoder of the second device to quantize the probe data.

[0171] For example, the relocation may include a first type of relocation and / or a second type of relocation.

[0172] For example, the first manner of rearrangement may include at least one of the following: when the intermediate data is intermediate data corresponding to lighting data, the first manner of rearrangement may include at least one of the following: adding a channel to the intermediate data corresponding to the lighting data, converting the intermediate data corresponding to the lighting data to a data storage format of the first device, or performing a dimensional transformation on the intermediate data corresponding to the lighting data.

[0173] It should be noted that the "intermediate data corresponding to the illumination data" in "converting intermediate data corresponding to the illumination data into a data storage format of the first device" may be original intermediate data in the bitstream corresponding to the illumination data, or intermediate data corresponding to illumination data obtained through domain transformation, or intermediate data corresponding to dequantized illumination data, or intermediate data corresponding to illumination data to which channels have been added, or intermediate data corresponding to illumination data obtained through dimensional transformation.

[0174] It should be noted that the "intermediate data corresponding to the illumination data" in "performing a dimensional transformation on intermediate data corresponding to the illumination data" may be original intermediate data in the bitstream corresponding to the illumination data, or intermediate data corresponding to illumination data obtained through domain transformation, or intermediate data corresponding to dequantized illumination data, or intermediate data corresponding to illumination data to which a channel has been added, or intermediate data corresponding to illumination data whose data format has been converted to the data storage format of the first device.

[0175] For example, when the intermediate data is intermediate data corresponding to visibility data and the intermediate data corresponding to the visibility data includes multiple channels, the first manner of rearrangement includes at least one of the following: combining the channels, converting the intermediate data corresponding to the visibility data to a data storage format of the first device, or performing a dimensional transformation on the intermediate data corresponding to the visibility data.

[0176] It should be noted that the "intermediate data corresponding to the visibility data" in "converting intermediate data corresponding to the visibility data into a data storage format of the first device" may be original intermediate data in the bitstream corresponding to the visibility data, or intermediate data corresponding to visibility data obtained through domain transformation, or intermediate data corresponding to dequantized visibility data, or intermediate data corresponding to visibility data to which channels have been added, or intermediate data corresponding to visibility data obtained through dimensional transformation.

[0177] It should be noted that the "intermediate data corresponding to the visibility data" in "performing a dimensional transformation on intermediate data corresponding to the visibility data" may be original intermediate data in the bitstream corresponding to the visibility data, or intermediate data corresponding to visibility data obtained through domain transformation, or intermediate data corresponding to dequantized visibility data, or intermediate data corresponding to visibility data to which channels have been added, or intermediate data corresponding to visibility data whose data format has been converted to the data storage format of the first device.

[0178] For example, the intermediate data may be data on a YUV plane, and the second method of rearrangement may be used to extract data from the YUV plane and arrange the data extracted from the YUV plane into a two-dimensional picture.

[0179] It should be noted that when the intermediate data is not data on a YUV plane, the step of performing the second type of rearrangement on the intermediate data may not be performed.

[0180] It should be noted that when the data extracted from the YUV plane are the transform base and transform coefficients, the rearrangement in the second manner may not include arranging the data extracted from the YUV plane into a two-dimensional picture.

[0181] Below we describe several schemes that combine domain transformation with inverse quantization and / or rearrangement.

[0182] FIG. 4a is a diagram of an example of a decoding procedure.

[0183] In the embodiment in Fig. 4a(1), a method of combining domain transformation and inverse quantization is described. See Fig. 4a(1). For example, after a bitstream is obtained, the bitstream may be decoded to obtain intermediate data. Then, the intermediate data is inverse quantized to obtain transformed data 1, and a domain transformation is performed on the transformed data 1 to obtain probe data.

[0184] In the embodiment in Fig. 4a(2), a scheme of combining domain transformation and rearrangement is described. See Fig. 4a(2). For example, after a bitstream is obtained, the bitstream may be decoded to obtain intermediate data. Then, rearrangement is performed on the intermediate data to obtain transformed data 2, and domain transformation is performed on the transformed data 2 to obtain probe data. The rearrangement in Fig. 4a(2) may include a first scheme of rearrangement and / or a second scheme of rearrangement.

[0185] In the embodiment in Fig. 4a(3), a method of combining domain transformation, inverse quantization and rearrangement is described. See Fig. 4a(3). For example, after a bitstream is obtained, the bitstream may be decoded to obtain intermediate data. Then, rearrangement is performed on the intermediate data to obtain transformed data 2, the transformed data 2 is inverse quantized to obtain transformed data 3, and a domain transformation is performed on the transformed data 3 to obtain probe data.

[0186] In the embodiment in Fig. 4a(4), a scheme of combining domain transformation, inverse quantization and rearrangement in a first manner is described. See Fig. 4a(4). For example, after a bitstream is obtained, the bitstream may be decoded to obtain intermediate data. Then, the intermediate data is inverse quantized to obtain transformed data 1, the first manner of rearrangement is performed on the transformed data 1 to obtain transformed data 4, and the domain transformation is performed on the transformed data 4 to obtain probe data.

[0187] In the embodiment in Fig. 4a(5), another manner of combining domain transformation, inverse quantization and rearrangement is described. See Fig. 4a(5). For example, after a bitstream is obtained, the bitstream may be decoded to obtain intermediate data. Then, a second manner of rearrangement is performed on the intermediate data to obtain transformed data 5, the transformed data 5 is inverse quantized to obtain transformed data 6, a first manner of rearrangement is performed on the transformed data 6 to obtain transformed data 7, and a domain transformation is performed on the transformed data 7 to obtain probe data.

[0188] FIG. 4b is a diagram of an example of a decoding procedure.

[0189] In the embodiment in Fig. 4b(1), a scheme of combining domain transformation and inverse quantization is described. See Fig. 4b(1). For example, after a bitstream is obtained, the bitstream may be decoded to obtain intermediate data. Then, a domain transformation is performed on the intermediate data to obtain transformed data 8, and the transformed data 8 is inverse quantized to obtain probe data.

[0190] In the embodiment in Fig. 4b(2), a scheme of combining domain transformation, inverse quantization and rearrangement is described. See Fig. 4b(2). For example, after a bitstream is obtained, the bitstream may be decoded to obtain intermediate data. Then, rearrangement is performed on the intermediate data to obtain transformed data 2, a domain transformation is performed on the transformed data 2 to obtain transformed data 9, and the transformed data 9 is inverse quantized to obtain probe data. The rearrangement in Fig. 4b(2) may include a first scheme of rearrangement and / or a second scheme of rearrangement.

[0191] In the embodiment in Fig. 4b(3), a scheme of combining domain transformation, inverse quantization and rearrangement is described. See Fig. 4b(3). For example, after a bitstream is obtained, the bitstream may be decoded to obtain intermediate data. Then, a domain transformation is performed on the intermediate data to obtain transformed data 8, a first manner of rearrangement is performed on the transformed data 8 to obtain transformed data 10, and the transformed data 10 is inverse quantized to obtain probe data.

[0192] In the embodiment in Fig. 4b(4), a scheme of combining domain transformation, inverse quantization and rearrangement is described. See Fig. 4b(4). For example, after a bitstream is obtained, the bitstream may be decoded to obtain intermediate data. Then, a second scheme of rearrangement is performed on the intermediate data to obtain transformed data 5, a domain transformation is performed on the transformed data 5 to obtain transformed data 11, a first scheme of rearrangement is performed on the transformed data 11 to obtain transformed data 12, and the transformed data 12 is inverse quantized to obtain probe data.

[0193] FIG. 4c(1) to FIG. 4c(6) are diagrams showing an example of a decoding procedure.

[0194] In the embodiment in Fig. 4c(1), a scheme of combining domain transformation and rearrangement is described. See Fig. 4c(1). For example, after a bitstream is obtained, the bitstream may be decoded to obtain intermediate data. Then, a domain transformation is performed on the intermediate data to obtain transformed data 8, and a first manner of rearrangement is performed on the transformed data 8 to obtain probe data.

[0195] In the embodiment in Fig. 4c(2), a method of combining domain transformation, rearrangement and inverse quantization is described. See Fig. 4c(2). For example, after a bitstream is obtained, the bitstream may be decoded to obtain intermediate data. Then, the intermediate data is inverse quantized to obtain transformed data 1, a domain transformation is performed on the transformed data 1 to obtain transformed data 13, and a first method of rearrangement is performed on the transformed data 13 to obtain probe data.

[0196] In the embodiment in Fig. 4c(3), a scheme of combining domain transformation, rearrangement and inverse quantization is described. See Fig. 4c(3). For example, after a bitstream is obtained, the bitstream may be decoded to obtain intermediate data. Then, a domain transformation is performed on the intermediate data to obtain transformed data 8, the transformed data 8 is inverse quantized to obtain transformed data 14, and a first manner of rearrangement is performed on the transformed data 14 to obtain probe data.

[0197] In the embodiment in Fig. 4c(4), a scheme of combining domain transformation and rearrangement is described. See Fig. 4c(4). For example, after a bitstream is obtained, the bitstream may be decoded to obtain intermediate data. Then, a second scheme of rearrangement is performed on the intermediate data to obtain transformed data 5, a domain transformation is performed on the transformed data 5 to obtain transformed data 11, and a first scheme of rearrangement is performed on the transformed data 11 to obtain probe data.

[0198] In the embodiment in Fig. 4c(5), a scheme of combining domain transformation, rearrangement and inverse quantization is described. See Fig. 4c(5). For example, after a bitstream is obtained, the bitstream may be decoded to obtain intermediate data. Then, a second scheme of rearrangement is performed on the intermediate data to obtain transformed data 5, the transformed data 5 is inverse quantized to obtain transformed data 6, a domain transformation is performed on the transformed data 6 to obtain transformed data 15, and a first scheme of rearrangement is performed on the transformed data 15 to obtain probe data.

[0199] In the embodiment in Fig. 4c(6), a method of combining domain transformation, rearrangement and inverse quantization is described. See Fig. 4c(6). For example, after a bitstream is obtained, the bitstream may be decoded to obtain intermediate data. Then, a second method of rearrangement is performed on the intermediate data to obtain transformed data 5, a domain transformation is performed on the transformed data 5 to obtain transformed data 11, the transformed data 11 is inverse quantized to obtain transformed data 16, and a first method of rearrangement is performed on the transformed data 16 to obtain probe data.

[0200] It should be noted that the domain transformations in Figures 4a(1)-4a(5), 4b(1)-4b(4), and 4c(1)-4c(6) may include at least one of the following: denormalization, transformation from a nonlinear domain to a linear domain, transformation from a YUV domain to an RGB domain, transformation from an XYZ domain to an RGB domain, and transformation from a Lab domain to an RGB domain. It should be understood that this application may further include other types of domain transformations. This is not limited in this application. Furthermore, this application does not limit the number of times to perform the various types of domain transformations in Figures 4a(1)-4a(5), 4b(1)-4b(4), and 4c(1)-4c(6) and the order of performing the different types of domain transformations.

[0201] It should be noted that this application does not limit the number of times or the order in which the inverse quantization is performed in Figures 4a(1) to 4a(5), 4b(1) to 4b(4), and 4c(1) to 4c(6).

[0202] It should be noted that the first type of rearrangement in Figures 4a(1)-4a(5), 4b(1)-4b(4), and 4c(1)-4c(6) may include at least one of the following: adding a channel, combining a channel, converting data to the data storage format of the first device, and dimension conversion. Furthermore, this application does not limit the number of times the different types of first type of rearrangement in Figures 4a(1)-4a(5), 4b(1)-4b(4), and 4c(1)-4c(6) are performed and the order of performing the different types of first type of rearrangement.

[0203] It should be understood that the decoding procedures of FIG. 3b and FIG. 4a-FIG. 4c(1)-FIG. 4c(6) correspond to the encoding procedures of the encoder of the second device.

[0204] In a possible manner, after obtaining the bitstream, the decoder of the first device may decode the bitstream to obtain bitstream structure information and attribute data. For example, the bitstream structure information may include, but is not limited to, the number of probes, the position, length and data format of intermediate data corresponding to illumination data, the execution order of data format conversion types corresponding to illumination data, the position, length and data format of intermediate data corresponding to visibility data, the execution order of data format conversion types corresponding to visibility data, the position, length and data format of the first attribute data, and the position, length and data format of the second attribute data. This is not limited in this application. The data format conversion type may include various data format conversion types used by the second device during encoding, such as corresponding various first scheme rearrangement types (e.g., discarding data of some channels, channel splitting, precision conversion, and dimension conversion), various normalization types (e.g., maximum normalization, fixed parameter normalization, min-max normalization, and z-score), various domain conversion types (e.g., linear domain to nonlinear domain conversion, RGB domain to YUV domain conversion, RGB domain to XYZ domain conversion, RGB domain to Lab domain conversion, and pixel conversion). The first attribute data may include a quantization parameter, a domain transformation parameter, and a rearrangement parameter. The domain transformation parameter may include a normalization parameter, an exponential transformation parameter, a PQ transformation parameter, an HLG transformation parameter, and a color space transformation parameter. The first attribute data may include a quantization parameter, a domain transformation parameter, a transformation parameter from a color domain to a transformation domain, and various quantization types (e.g., uniform quantization and non-uniform quantization). The second scheme relocation type includes a second relocation type used by the second device during encoding, e.g., padding to YUV planes. This is not limited in this application. The first attribute data may include a quantization parameter, a domain transformation parameter, and a rearrangement parameter. The domain transformation parameter may include a normalization parameter, an exponential transformation parameter, a PQ transformation parameter, an HLG transformation parameter, and a color space transformation parameter. This is not limited in this application.

[0205] In this way, the decoder of the first device may perform corresponding data format conversion on the intermediate data based on the bitstream structure information and the first attribute data obtained through decoding, to restore the probe data from the intermediate data.

[0206] It should be understood that the bitstream structure information may include more or less information than that listed above, and this is not a limitation of this application.

[0207] Fig. 4d is a diagram of an example of the structure of a bitstream. The bitstream shown in Fig. 4d includes intermediate data corresponding to visibility data, intermediate data corresponding to lighting data, attribute data, and bitstream structure information.

[0208] It should be understood that the bitstream structure information may include more or less information than the above information shown in Figure 4d, and this is not a limitation of this application.

[0209] In a possible manner, the decoder of the first device may pre-obtain information about the data conversion format in the encoding process of the encoder of the second device, including but not limited to the execution order of the data format conversion types corresponding to the illumination data, the execution order of the data format conversion types corresponding to the visibility data, the first attribute data, etc. This is not limited in this application. In this case, the bitstream may not include the bitstream structure information and the attribute data.

[0210] In this way, after obtaining the intermediate data, the decoder of the first device may perform corresponding data format conversion on the intermediate data based on information regarding the data conversion format in the encoding process obtained in advance, to restore the probe data from the intermediate data.

[0211] In the following, the decoding procedure shown in FIG. 4c(5) is used as an example to describe in detail the process of obtaining illumination data through decoding.

[0212] FIG. 5 is a diagram of an example of a decoding procedure. The decoding procedure may include the following steps: S501: Obtain a bitstream. S502: Perform HEVC decoding. S503: Perform data extraction. S504: Perform data placement. S505: Perform inverse quantization. S506: Perform YUV domain to RGB domain conversion. S507: Perform exponential domain to linear domain conversion. S508: Perform inverse normalization. S509: Perform format conversion. In the embodiment in FIG. 5, the intermediate data is data represented by Y, U and V values. The conversion data 5 includes conversion data 5a and conversion data 5b, and the conversion data 15 includes conversion data 15a, conversion data 15b and conversion data 15c.

[0213] S501: Obtain a bitstream, that is, obtain a bitstream, where the bitstream includes intermediate data corresponding to illumination data encoded by a second device.

[0214] For example, the intermediate data included in the bitstream corresponds to illumination data for K probes.

[0215] S502: Perform HEVC decoding, i.e., perform HEVC decoding on the bitstream obtained in S501 to obtain intermediate data corresponding to lighting data, where the intermediate data is data on a YUV plane, and the data padded to the YUV plane is represented by Y, U and V values.

[0216] For example, HEVC decoding may be performed on the bitstream, and intermediate data corresponding to the illumination data of the K probes may be obtained from the bitstream through decoding.

[0217] For example, whether the intermediate data is data on a YUV plane may be determined based on the relocation parameters. In the embodiment in Fig. 5, the intermediate data is data on a YUV plane.

[0218] For example, the type of the YUV plane may be determined based on the relocation parameters, for example, the YUV plane may be a YUV 444 plane, a YUV 420 plane, or another type of YUV plane, which is not limited in this application.

[0219] For example, the precision of the data on the YUV plane may be determined based on the relocation parameters. For example, the precision of the data on the YUV plane may be 16 bits or 12 bits. This is also not limited in this application.

[0220] For example, the precision of the valid data among the data padded to the YUV plane may be determined based on the relocation parameters. For example, the precision of the valid data among the data padded to the YUV plane is equal to or less than the precision of the data on the YUV plane. For example, the precision of the data on the YUV plane may be 16 bits, and the precision of the valid data among the data padded to the YUV plane may be 12 bits (the other 4 bits are padded with 0). In another example, the precision of the data on the YUV plane may be 12 bits, and the precision of the valid data among the data padded to the YUV plane may be 12 bits.

[0221] For example, the representation format of the data padded in the YUV plane may be determined based on the relocation parameters. In the embodiment in Fig. 5, the data padded in the YUV plane may be data of three channels of Y, U and V, and each of the data of the three channels of Y, U and V is represented by a Y, U and V value.

[0222] S503: Perform data extraction, ie, extract data from the YUV plane obtained in S502 to obtain transformed data 5a.

[0223] For example, the YUV plane may include a Y plane, a U plane, and a V plane. When the YUV plane is a YUV 420 plane, the Y plane:U plane:V plane=4:1:1. When the YUV plane is a YUV 444 plane, the Y plane:U plane:V plane=1:1:1.

[0224] For example, Y channel data is padded into the Y plane. In this manner, the Y channel data may be extracted from the Y plane. For example, U channel data is padded into the U plane. In this manner, the U channel data may be extracted from the U plane. For example, V channel data is padded into the V plane. In this manner, the V channel data may be extracted from the V plane. It should be noted that when data is extracted from the YUV plane, invalid bits of precision of the data may be discarded.

[0225] In this way, three channels of data, Y, U and V, may be extracted from the YUV plane, and the three channels of data, Y, U and V, are called transformed data 5a.

[0226] S504: Execute data arrangement, that is, arrange the transformed data 5a obtained in S503 into a two-dimensional picture to obtain transformed data 5b.

[0227] For example, the transformation data 5a may be arranged in a two-dimensional picture, where each pixel of the two-dimensional picture includes three channels, Y, U and V, and the data for each channel is represented by Y, U and V values. Each probe corresponds to multiple pixels in the two-dimensional picture, and the intermediate data corresponding to the illumination data for each probe includes multiple pixels.

[0228] For example, the two-dimensional picture obtained by arranging the transformed data 5a may be called transformed data 5b, in other words, the transformed data 5b is a two-dimensional picture in which a single pixel includes three channels, Y, U and V, and the data of each channel is represented by Y, U and V values.

[0229] When the YUV plane is a YUV 444 plane, the three channels of Y, U and V in the converted data 5b are represented by YUV 444, i.e., each pixel in the converted data 5b corresponds to one Y channel, one U channel and one V channel. When the YUV plane is a YUV 420 plane, the three channels of Y, U and V in the converted data 5b are represented by YUV 420, and each pixel in the converted data 5b corresponds to one Y channel, and each of four pixels corresponds to one U channel and one V channel.

[0230] It should be understood that in S503, the data padded to the YUV plane may be data of three channels of X, Y and Z, and each data of the three channels of X, Y and Z is represented by X, Y and Z values. In this way, the obtained conversion data 5b is a two-dimensional picture in which a single pixel includes three channels of X, Y and Z. Furthermore, in S503, the data padded to the YUV plane may be data of three channels of L, a and b, and each data of the three channels of L, a and b is represented by L, a and b values. In this way, the obtained conversion data 5b is a two-dimensional picture in which a single pixel includes three channels of L, a and b. This is not limited in this application. This application provides an explanation by using an example in which the conversion data 5b is a two-dimensional picture in which a single pixel includes three channels of Y, U and V.

[0231] S505: Perform inverse quantization, that is, inverse quantize the transformed data 5b obtained in S504 to obtain transformed data 6.

[0232] For example, after the transformed data 5b is obtained, the transformed data 5b may be dequantized based on the quantization parameter to obtain the transformed data 6.

[0233] For example, for a pixel in the transformed data 5b, the Y channel data, the U channel data, and the V channel data of the pixel are dequantized separately based on the quantization parameter. In this manner, the data of each of the three channels Y, U, and V of each pixel in the transformed data 5b may be dequantized to obtain the transformed data 6. For example, after being dequantized, the data of each channel of each pixel in the transformed data 5b may be transformed into the range of [0,1].

[0234] For example, the transformed data 6 is a two-dimensional picture in which a single pixel includes three channels, Y, U and V, and the data for each of the three channels, Y, U and V, of each pixel in the two-dimensional picture is dequantized data.

[0235] When the YUV plane is a YUV 444 plane, the three channels of Y, U, and V in the transformed data 6 are represented by YUV 444. When the YUV plane is a YUV 420 plane, the three channels of Y, U, and V in the transformed data 6 are represented by YUV 420.

[0236] S506: Implement conversion from the YUV domain to the RGB domain, that is, implement conversion from the YUV domain to the RGB domain on the converted data 6 acquired in S505 to obtain converted data 15a.

[0237] For example, the color space conversion parameters may be in a color space format.

[0238] For example, when it is determined based on the color space format that the three channels of Y, U and V in the conversion data 6 are represented by YUV 444, the three channels of Y, U and V data of each pixel in the conversion data 6 are converted into three channels of R, G and B data in the manner of conversion from YUV 444 color space to RGB color space to obtain conversion data 15a.

[0239] For example, when it is determined based on the color space format that the three channels of Y, U and V in the conversion data 6 are represented by YUV 420, the three channels of Y, U and V data of each pixel in the conversion data 6 are converted into three channels of R, G and B data in the manner of conversion from YUV 420 color space to RGB color space to obtain conversion data 15a.

[0240] Thus, the obtained transformed data 15a is a two-dimensional picture in which a single pixel contains three channels of R, G and B, and the data of each channel in the two-dimensional picture is dequantized data.

[0241] It should be understood that when it is determined based on the color space format that the conversion data 6 is a two-dimensional picture in which a single pixel includes three channels of X, Y, and Z, the conversion from the YUV domain to the RGB domain in S506 may be replaced with a conversion from the XYZ domain to the RGB domain. Alternatively, when it is determined based on the color space format that the conversion data 6 is a two-dimensional picture in which a single pixel includes three channels of L, a, and b, the conversion from the YUV domain to the RGB domain in S506 may be replaced with a conversion from the Lab domain to the RGB domain. This is not limited in this application.

[0242] S507: Perform a transformation from the exponential domain to the linear domain, that is, perform a transformation from the exponential domain to the linear domain on the transformed data 15a obtained in S506 to obtain transformed data 15b.

[0243] For example, when the domain transformation parameters include an exponential transformation parameter, the transformed data 15a may be determined to belong to the exponential domain, in which case, based on the exponential transformation parameter, a transformation from the exponential domain to the linear domain may be performed on the transformed data 15a to obtain the transformed data 15b.

[0244] For example, for a pixel in the transformation data 15a, the extraction of the nth root (n is an exponential transformation parameter) is performed separately for each of the R, G, and B three channel data of the pixel to perform a transformation from the exponential domain to the linear domain for each of the R, G, and B three channel data of the pixel. In this manner, a transformation from the exponential domain to the linear domain may be performed for each of the R, G, and B three channel data of each pixel in the two-dimensional picture. In other words, the transformation data 15b is a two-dimensional picture in which a single pixel includes three R, G, and B channels, and each of the R, G, and B three channel data of each pixel belongs to the linear domain.

[0245] It should be understood that when the domain transformation parameters include PQ transformation parameters, the transformation data 15a may be determined to belong to the PQ domain. In this case, S506 may be replaced with performing a transformation from the PQ domain to a linear domain on the transformation data 15a to obtain the transformation data 15b. For example, the PQ transformation parameters may be parameters of a PQ curve. For a pixel in the transformation data 15a, an inverse mapping is separately performed on each data of the three channels R, G, and B of the pixel through the PQ curve to perform a transformation from the PQ domain to a linear domain on each data of the three channels R, G, and B of the pixel. In this way, a transformation from the PQ domain to a linear domain may be performed on each data of the three channels R, G, and B of each pixel in the two-dimensional picture. In this case, the transformation data 15b is a two-dimensional picture in which a single pixel includes three channels R, G, and B, and each data of the three channels R, G, and B of each pixel belongs to a linear domain.

[0246] It should be understood that when the domain transformation parameters include HLG transformation parameters, it may be determined that the transformation data 15a belongs to the HLG domain. In this case, S506 may be replaced with performing a transformation from the HLG domain to a linear domain on the transformation data 15a to obtain the transformation data 15b. For example, the HLG transformation parameters may be parameters of an HLG curve. For a pixel in the transformation data 15a, an inverse mapping is separately performed on each data of the R, G, and B three channels of the pixel through the HLG curve to perform a transformation from the HLG domain to a linear domain on each data of the R, G, and B three channels of the pixel. In this way, a transformation from the HLG domain to a linear domain may be performed on each data of the R, G, and B three channels of each pixel in the transformation data 15a. In this case, the transformation data 15b is a two-dimensional picture in which a single pixel includes three channels of R, G, and B, and each data of the R, G, and B three channels of each pixel belongs to a linear domain.

[0247] S508: Perform denormalization, that is, perform denormalization on the transformed data 15b obtained in S507 to obtain transformed data 15c.

[0248] For example, de-normalization may be performed on transformed data 15b based on the normalization parameter to obtain transformed data 15c.

[0249] For example, the normalization parameter may have a type identifier, and the type identifier may uniquely identify the normalization type. When it is determined whether the fixed parameter normalization, the adaptive normalization, or the maximum normalization is performed in the encoding process based on the type identifier of the normalization parameter, for one pixel in the conversion data 15b, each data of the three channels of R, G, and B of the pixel may be divided by the normalization parameter to perform inverse normalization on each data of the three channels of R, G, and B of the pixel. Further, in this way, inverse normalization may be performed on each data of the three channels of R, G, and B of each pixel in the conversion data 15b.

[0250] For example, when it is determined that min-max normalization is performed in the encoding process based on the type identification information of the normalization parameter, in a possible manner, the normalization parameter may include a maximum value and a minimum value corresponding to the R channel, a maximum value and a minimum value corresponding to the G channel, and a maximum value and a minimum value corresponding to the B channel. For the R channel data of the pixel in the conversion data 15b, a difference Q1 between the maximum value and the minimum value corresponding to the R channel may be calculated, a product C1 of the R channel data of the pixel and the difference Q1 is calculated, and a sum of the product C1 and the minimum value is calculated to obtain the R channel data of the pixel obtained through denormalization. In this way, denormalization can be performed on the R channel data of the pixel. For the G channel data and the B channel data of the pixel, denormalization may be performed in the above manner to perform denormalization on each of the three channels of R, G, and B of the pixel in the two-dimensional picture. Similarly, in this way, denormalization may be performed on each of the three channels of R, G, and B of the pixel in the two-dimensional picture.

[0251] For example, when it is determined that min-max normalization is performed in the encoding process based on the type identification information of the normalization parameter, in a possible manner, the normalization parameter may include a maximum value and a minimum value corresponding to all channels. For the R channel data of the pixel in the conversion data 15b, the difference Q2 between the maximum value and the minimum value corresponding to all channels may be calculated, the product C2 of the R channel data of the pixel and the difference Q2 is calculated, and the sum of the product C2 and the minimum value is calculated to obtain the R channel data of the pixel obtained through denormalization, thereby performing denormalization on the R channel data of the pixel. For the G channel data and B channel data of the pixel, denormalization may be performed in the above manner to perform denormalization on each of the three channels of R, G, and B of the pixel in the conversion data 15b. Similarly, in this manner, denormalization may be performed on each of the three channels of R, G, and B of each pixel in the conversion data 15b.

[0252] For example, when it is determined based on the type identification information of the normalization parameter that the z-score normalization is performed in the decoding process, the normalization parameter may include a mean value and a standard deviation. For a pixel in the transformed data 15b, the data of the R channel of the pixel is multiplied by the standard deviation to obtain a product H, and the product H is added to the mean value to obtain the data obtained through the inverse normalization of the R channel of the pixel. In this way, the data obtained through the inverse normalization of the G channel of the pixel and the data obtained through the inverse normalization of the B channel of the pixel may be obtained. Similarly, in this way, the data obtained through the inverse normalization of each of the data of the three channels R, G and B of each pixel in the transformed data 15b may be determined.

[0253] Thus, the obtained conversion data 15c is also a two-dimensional picture in which a single pixel contains three channels of R, G and B, and the data of each channel of each pixel is obtained through inverse normalization.

[0254] S509: Perform format conversion, that is, convert the data format of the converted data 15c obtained in S508 into the data storage format of the first device to obtain the illumination data.

[0255] For example, the data storage format of the first device may be obtained, and then the data format of the conversion data 15c may be converted based on the data storage format of the first device to obtain the lighting data.

[0256] For example, the data storage format of the first device is RGB16F (R, B, and G channels are each represented by a 16-bit floating-point number), and the data format of the conversion data 15c is RGB12 (R, B, and G channels are each represented by a 12-bit unsigned type). In this case, the RGB format of each pixel in the two-dimensional picture may be converted from RGB12 to RGB16F.

[0257] In this way, the acquired illumination data is a two-dimensional picture where a single pixel contains three channels, R, G and B, and each probe corresponds to multiple pixels in the two-dimensional picture, and the illumination data for each probe contains multiple pixels.

[0258] It should be noted that the illumination data acquired in S509 is a two-dimensional picture in which a single pixel includes three channels of R, G and B, and the data of the A channel may be added to the illumination data based on requirements. Thus, the illumination data acquired is a two-dimensional picture including four channels of R, G, B and A, which is not limited in this application.

[0259] It should be understood that since the data storage formats of the first device and the second device are different, the data format of the probe data acquired by the first device through decoding may be the same as or different from the data format of the probe data encoded by the second device, which is not a limitation in this application.

[0260] It should be noted that the order of the four steps S505 to S508 may be adjusted randomly, which is not limited in this application.

[0261] FIG. 6 is a diagram of an example of a decoding procedure. The decoding procedure may include the following steps: S601: Obtain a bitstream; S602: Perform HEVC decoding; S603: Perform data extraction; S604: Perform inverse quantization; S605: Perform YUV domain to RGB domain conversion; S606: Perform exponential domain to linear domain conversion; S607: Perform inverse normalization; S608: Perform format conversion. In the embodiment in FIG. 6, the intermediate data is data represented by spherical harmonic basis coefficients. The transformation data 15 includes transformation data 15a, transformation data 15b, and transformation data 15c.

[0262] S601: Obtain a bitstream, that is, obtain a bitstream, where the bitstream includes intermediate data corresponding to illumination data encoded by a second device.

[0263] S602: Perform HEVC decoding, i.e., perform HEVC decoding on the bitstream obtained in S601 to obtain intermediate data corresponding to illumination data, where the intermediate data is data on a YUV plane, and the data padded to the YUV plane is represented by spherical harmonic basis coefficients.

[0264] For example, for S602, please refer to the description of S502, and the details will not be described again here.

[0265] In the embodiment in Figure 6, the intermediate data is data on a YUV plane, and the data padded to the YUV plane may be data of three channels, Y, U and V, and each of the data of the three channels, Y, U and V, is represented by spherical harmonic basis coefficients.

[0266] It should be noted that the data of each of the three channels Y, U and V may alternatively be represented by spherical wavelet basis coefficients or transform coefficients corresponding to other transform bases, which is not limited in this application. In the following, an explanation is provided by using an example in which the data padded in the YUV plane of each of the three channels Y, U and V is represented by spherical harmonic basis coefficients.

[0267] S603: Perform data extraction, ie, extract data from the YUV plane obtained in S602 to obtain transformed data 5.

[0268] For example, the data extracted from the YUV plane may include transformation data 5 corresponding to the illumination data of each probe. The transformation data 5 corresponding to the illumination data of each probe may include spherical harmonic basis coefficients for three channels: Y, U and V, each channel including x spherical harmonic basis coefficients.

[0269] In the rendering process, the first device may determine the shading effect of the object in the three-dimensional scene based on the spherical harmonic basis and the corresponding spherical harmonic basis coefficient. Therefore, after the data is extracted from the YUV plane, the data extracted from the YUV plane is directly dequantized without being placed into a two-dimensional picture, that is, S604 is directly executed.

[0270] S604: Perform inverse quantization, that is, inverse quantize the transformed data 5 obtained in S603 to obtain transformed data 6.

[0271] For example, for the transformation data 5 corresponding to the illumination data of the probe, each spherical harmonic basis coefficient of each of the three channels of Y, U and V in the transformation data 5 may be inverse quantized based on the quantization parameter to obtain the transformation data 6 corresponding to the illumination data of the probe. Each spherical harmonic basis coefficient of each of the three channels of Y, U and V in the transformation data 5 may be inverse quantized to the range of [0,1] through inverse quantization. Further, in this way, the transformation data 5 corresponding to the illumination data of all the probes may be inverse quantized to obtain the transformation data 6 corresponding to the illumination data of all the probes. The transformation data 6 corresponding to the illumination data of each probe may include spherical harmonic basis coefficients of the three channels of Y, U and V. Each channel includes x spherical harmonic basis coefficients, each spherical harmonic basis coefficient in each channel is a dequantized value, and each spherical harmonic basis coefficient in each channel belongs to [0,1].

[0272] S605: Implement conversion from the YUV domain to the RGB domain, that is, implement conversion from the YUV domain to the RGB domain on the converted data 6 acquired in S604 to obtain converted data 15a.

[0273] For example, for S605, please refer to the description of S506, and the details will not be described again here.

[0274] A YUV domain to RGB domain transformation is performed on the transformation data 6 corresponding to the illumination data of each probe to obtain transformation data 15a corresponding to the illumination data of each probe. The transformation data 15a corresponding to the illumination data of each probe may include spherical harmonic basis coefficients of three channels of R, G and B, where each channel includes x spherical harmonic basis coefficients.

[0275] S606: Perform a transformation from the exponential domain to the linear domain, that is, perform a transformation from the exponential domain to the linear domain on the transformed data 15a obtained in S605 to obtain transformed data 15b.

[0276] For example, for S606, please refer to the description of S507, and the details will not be described again here.

[0277] For example, the transformation data 15a corresponding to the illumination data of each probe is transformed from the exponential domain to the linear domain to obtain the transformation data 15b corresponding to the illumination data of each probe. The transformation data 15b corresponding to the illumination data of each probe may include spherical harmonic basis coefficients of three channels of R, G and B, each channel includes x spherical harmonic basis coefficients, and each spherical harmonic basis coefficient belongs to the linear domain.

[0278] S607: Perform inverse normalization, that is, perform min-max inverse normalization on the transformed data 15b obtained in S606 to obtain transformed data 15c.

[0279] In a possible manner, the normalization parameters include x normalization parameters (including maximum and minimum values) corresponding separately to x spherical harmonic bases in the R channel, x normalization parameters (including maximum and minimum values) corresponding separately to x spherical harmonic bases in the G channel, and x normalization parameters (including maximum and minimum values) corresponding separately to x spherical harmonic bases in the B channel. For example, a difference Q3 between the maximum value and the minimum value corresponding to the i-th (i is a value between 1 and x, and i is an integer) spherical harmonic base in the R channel may be calculated. Then, for the i-th spherical harmonic base coefficient in the R channel in the transformation data 15b corresponding to the illumination data of the probe, the product of the i-th spherical harmonic base coefficient and the difference Q3 is calculated to obtain a product value F5, and the product value F5 is added to the minimum value to obtain the value of the i-th spherical harmonic base coefficient obtained through inverse normalization in the R channel in the transformation data 15b corresponding to the illumination data of the probe. In this way, inverse normalization may be performed on the i-th spherical harmonic basis coefficient in the R channel in the transformation data 15b corresponding to the illumination data of each probe. Similarly, in the above manner of performing inverse normalization on the spherical harmonic basis coefficient of x in the R channel in the transformation data 15b corresponding to the illumination data of each probe, inverse normalization may be performed on the spherical harmonic basis coefficient of x in the G channel and the B channel in the transformation data 15b corresponding to the illumination data of each probe to obtain the transformation data 15c corresponding to the illumination data of all the probes. The transformation data 15c corresponding to the illumination data of each probe may also include spherical harmonic basis coefficients of three channels of R, G and B, each channel includes x spherical harmonic basis coefficients, and each spherical harmonic basis coefficient is a value obtained through inverse normalization.

[0280] In a possible manner, the normalization parameters include x normalization parameters (including maximum and minimum values) corresponding separately to x spherical harmonic bases shared by the three channels of R, G and B. A difference Q4 between the maximum value and the minimum value corresponding to the i-th (i is a value from 1 to x, and i is an integer) spherical harmonic base may be calculated. Then, for the i-th spherical harmonic base coefficient in the R channel in the transformation data 15b corresponding to the illumination data of the probe, the product of the i-th spherical harmonic base coefficient and the difference Q4 is calculated to obtain a product value F6, and the product value F6 is added to the minimum value to obtain the value of the i-th spherical harmonic base coefficient obtained through inverse normalization in the R channel in the transformation data 15b corresponding to the illumination data of the probe. In this way, inverse normalization may be performed on the i-th spherical harmonic base coefficient in the R channel in the transformation data 15b corresponding to the illumination data of each probe. Similarly, in the above manner of performing inverse normalization on the spherical harmonic basis coefficient of x in the R channel in the transformation data 15b corresponding to the illumination data of each probe, inverse normalization may be performed on the spherical harmonic basis coefficient of x in the G channel and the B channel in the transformation data 15b corresponding to the illumination data of each probe to obtain the transformation data 15c corresponding to the illumination data of each probe. The transformation data 15c corresponding to the illumination data of each probe may also include spherical harmonic basis coefficients of three channels of R, G and B, each channel includes x spherical harmonic basis coefficients, and each spherical harmonic basis coefficient is a value obtained through inverse normalization.

[0281] S608: Perform format conversion, that is, convert the data format of the converted data 15c obtained in S607 into the data storage format of the first device to obtain the illumination data.

[0282] For example, for S608, please refer to the description of 509. Details will not be described again here. The acquired illumination data of each probe includes spherical harmonic basis coefficients of three channels of R, G and B, and each channel includes x spherical harmonic basis coefficients.

[0283] It should be noted that the A channel data (which may include x spherical harmonic basis coefficients) may be added to the illumination data of each probe based on requirements. In this way, the acquired illumination data includes spherical harmonic basis coefficients of four channels of R, G, B and A, and each channel includes x spherical harmonic basis coefficients.

[0284] It should be noted that the order of the four steps S604 to S607 may be adjusted randomly, which is not limited in this application.

[0285] In the following, the decoding procedure shown in FIG. 4c(5) is used as an example to explain in detail the process of obtaining visibility data through decoding.

[0286] FIG. 7 is a diagram of an example of a decoding procedure. The decoding procedure may include the following steps: S701: get a bitstream; S702: perform HEVC decoding; S703a: perform data extraction; S703b: perform data extraction; S704a: perform data placement; S704b: perform data placement; S705a: perform inverse quantization; S705b: perform inverse quantization; S706a: perform inverse normalization; S706b: perform inverse normalization; S707a: perform format conversion; S707b: perform format conversion. In the embodiment in FIG. 7, the visibility data may include distance data and the square of the distance data, and the intermediate data is represented by Y, U and V values. The transformed data 5 includes transformed data 5a1, transformed data 5a2, transformed data 5b1 and transformed data 5b2, the transformed data 6 includes transformed data 6a and transformed data 6b, and the transformed data 15 includes transformed data 15a and transformed data 15b.

[0287] S701: Obtain a bitstream, that is, obtain a bitstream, where the bitstream includes intermediate data corresponding to visibility data encoded by a second device.

[0288] For example, the bitstream may include intermediate data corresponding to distance data and intermediate data corresponding to a square of the distance data, the intermediate data corresponding to the distance data corresponding to K probes, and the intermediate data corresponding to the square of the distance data corresponding to K probes.

[0289] S702: Perform HEVC decoding, that is, perform HEVC decoding on the bitstream obtained in S701 to obtain intermediate data corresponding to visibility data, where the intermediate data is represented by Y, U and V values.

[0290] For example, HEVC decoding is performed on the bitstream to obtain intermediate data corresponding to the distance data of the K probes and intermediate data corresponding to the square of the distance data of the K probes.

[0291] For example, for S702, please refer to the description of S502, and the details will not be described again here.

[0292] In the embodiment in FIG. 7, the intermediate data is data on the YUV plane.

[0293] The intermediate data corresponding to the distance data of the K probes may be data on a YUV plane (hereinafter, which may be referred to as the first YUV plane), and the intermediate data corresponding to the square of the distance data of the K probes may be data on another YUV plane (hereinafter, which may be referred to as the second YUV plane).

[0294] For example, for the description of the first YUV plane and the second YUV plane, please refer to the description of the YUV plane in the above embodiment, and the details will not be described again here.

[0295] In the embodiment of FIG. 7, the data padded into the first YUV plane may be three channels of data, Y, U and V, each of which is represented by a Y, U and V value, and the data padded into the second YUV plane may be three channels of data, Y, U and V, each of which is represented by a Y, U and V value.

[0296] S703a: Perform data extraction, ie, extract data from the first YUV plane obtained in S702, to obtain transformed data 5a1.

[0297] S703b: Perform data extraction, ie, extract data from the second YUV plane obtained in S702, to obtain transformed data 5a2.

[0298] For example, for S703a and S703b, please refer to the description of S503, and the details will not be described again here.

[0299] It should be noted that S703a and S703b may be performed in parallel or in series, which is not a limitation in this application.

[0300] S704a: Perform data arrangement, ie, arrange the transformed data 5a1 obtained in S703a into a two-dimensional picture to obtain transformed data 5b1.

[0301] S704b: Perform data arrangement, ie, arrange the transformed data 5a2 obtained in S703b into a two-dimensional picture to obtain transformed data 5b2.

[0302] For example, for S704a and S704b, please refer to the description of S504, and the details will not be described again here.

[0303] It should be noted that S704a and S704b may be performed in parallel or in series, which is not a limitation in this application.

[0304] S705a: Perform inverse quantization, that is, inverse quantize the transformed data 5b1 obtained in S704a to obtain transformed data 6a.

[0305] S705b: Perform inverse quantization, that is, inverse quantize the transformed data 5b2 obtained in S704b to obtain transformed data 6b.

[0306] For example, for S705a and S705b, please refer to the description of S505, and the details will not be described again here.

[0307] It should be noted that S705a and S705b may be performed in parallel or in series, which is not a limitation in this application.

[0308] S706a: Perform denormalization, that is, perform denormalization on the transformed data 6a obtained in S705a to obtain transformed data 15a.

[0309] S706b: Perform denormalization, that is, perform denormalization on the transformed data 6b obtained in S705b to obtain transformed data 15b.

[0310] For example, for S706a and S706b, please refer to the description of S508, and the details will not be described again here.

[0311] It should be noted that S706a and S706b may be performed in parallel or serially, which is not a limitation in this application.

[0312] S707a: Perform format conversion, ie, convert the data format of the converted data 15a obtained in S706a into the data storage format of the first device to obtain distance data.

[0313] S707b: Perform format conversion, ie, convert the data format of the converted data 15b obtained in S706b into the data storage format of the first device, to obtain the square of the distance data.

[0314] For example, for S707a and S707b, please refer to the description of S509, and the details will not be described again here.

[0315] It should be noted that S707a and S707b may be performed in parallel or in series, which is not a limitation in this application.

[0316] It should be noted that after the transformation data 15a and the transformation data 15b are obtained, the transformation data 15a may be used as the R channel, the transformation data 15b may be used as the G channel, and channel combination is performed on the transformation data 15a and the transformation data 15b to obtain the transformation data 15. The transformation data 15 may be a two-dimensional picture in which a single pixel includes two channels of R and G. Then, the data format of the transformation data 15 may be converted into the data storage format of the first device to obtain the distance data and the square of the distance data. It is clear that after the transformation data 5b1 and the transformation data 5b2 are obtained, the transformation data 5b1 may be used as the R channel, the transformation data 5b2 may be used as the G channel, and channel combination is performed on the transformation data 5b1 and the transformation data 5b2 to obtain the transformation data 5. The transformation data 5 may be a two-dimensional picture in which a single pixel includes two channels of R and G. Then, subsequent steps S705a, S706a and S707a (or S705b, S706b and S707b) are executed, which is not limited in this application.

[0317] FIG. 8 is a diagram of an example of a decoding procedure. The decoding procedure may include the following steps: S801: Obtain a bitstream. S802: Perform HEVC decoding. S803a: Perform data extraction. S803b: Perform data extraction. S804a: Perform inverse quantization. S804b: Perform inverse quantization. S805a: Perform inverse normalization. S805b: Perform inverse normalization. S806a: Perform format conversion. S806b: Perform format conversion. In the embodiment in FIG. 8, the visibility data may include distance data and the square of the distance data, and the intermediate data is represented by spherical harmonic basis coefficients. The transformation data 5 includes transformation data 5a1 and transformation data 5a2, the transformation data 6 includes transformation data 6a and transformation data 6b, and the transformation data 15 includes transformation data 15a and transformation data 15b.

[0318] S801: Obtain a bitstream, that is, obtain a bitstream, where the bitstream includes intermediate data corresponding to visibility data encoded by a second device.

[0319] For example, the bitstream may include intermediate data corresponding to distance data and intermediate data corresponding to a square of the distance data, the intermediate data corresponding to the distance data corresponding to K probes, and the intermediate data corresponding to the square of the distance data corresponding to K probes.

[0320] S802: Perform HEVC decoding, that is, perform HEVC decoding on the bitstream obtained in S801 to obtain intermediate data corresponding to visibility data, where the intermediate data is represented by Y, U and V values.

[0321] For example, HEVC decoding is performed on the bitstream to obtain intermediate data corresponding to the distance data of the K probes and intermediate data corresponding to the square of the distance data of the K probes.

[0322] For example, for S802, please refer to the description of S502, and the details will not be described again here.

[0323] In the embodiment in FIG. 8, the intermediate data is data on the YUV plane.

[0324] The intermediate data corresponding to the distance data of the K probes may be data on a YUV plane (hereinafter, which may be referred to as the first YUV plane), and the intermediate data corresponding to the square of the distance data of the K probes may be data on another YUV plane (hereinafter, which may be referred to as the second YUV plane).

[0325] For example, for the description of the first YUV plane and the second YUV plane, please refer to the description of the YUV plane in the above embodiment, and the details will not be described again here.

[0326] For example, in the embodiment in Fig. 7, the data padded in the first YUV plane may be data of three channels, Y, U and V, where each of the data of the three channels, Y, U and V, is represented by spherical harmonic basis coefficients (or spherical wavelet basis coefficients). The data padded in the second YUV plane may be data of three channels, Y, U and V, where each of the data of the three channels, Y, U and V, is represented by spherical harmonic basis coefficients (or spherical wavelet basis coefficients).

[0327] S803a: Perform data extraction, ie, extract data from the first YUV plane obtained in S802, to obtain transformed data 5a1.

[0328] S803b: Perform data extraction, ie, extract data from the second YUV plane obtained in S802, to obtain transformed data 5a2.

[0329] For example, for S803a and S803b, please refer to the description of S503, and the details will not be described again here.

[0330] It should be noted that S803a and S803b may be performed in parallel or in series, which is not a limitation in this application.

[0331] S804a: Perform inverse quantization, that is, inverse quantize the transformed data 5b1 obtained in S803a to obtain transformed data 6a.

[0332] S804b: Perform inverse quantization, that is, inverse quantize the transformed data 5b2 obtained in S803b to obtain transformed data 6b.

[0333] For example, for S804a and S804b, please refer to the description of S604, and the details will not be described again here.

[0334] It should be noted that S804a and S804b may be performed in parallel or in series, which is not a limitation in this application.

[0335] S805a: Perform denormalization, that is, perform denormalization on the transformed data 6a obtained in S804a to obtain transformed data 15a.

[0336] S805b: Perform denormalization, that is, perform denormalization on the transformed data 6b obtained in S804b to obtain transformed data 15b.

[0337] For example, for S805a and S805b, please refer to the description of S607, and the details will not be described again here.

[0338] It should be noted that S805a and S805b may be performed in parallel or serially, which is not a limitation in this application.

[0339] S806a: Perform format conversion, ie, convert the data format of the conversion data 15a obtained in S805a into the data storage format of the first device to obtain distance data.

[0340] S806b: Perform format conversion, ie, convert the data format of the conversion data 15b obtained in S805b into the data storage format of the first device, to obtain the square of the distance data.

[0341] For example, for S806a and S806b, please refer to the description of S607, and the details will not be described again here.

[0342] It should be noted that S806a and S806b may be performed in parallel or serially, which is not a limitation in this application.

[0343] It should be noted that after the transformation data 15a and the transformation data 15b are obtained, the transformation data 15a may be used as the R channel, and the transformation data 15b may be used as the G channel, and channel combination is performed on the transformation data 15a and the transformation data 15b to obtain the transformation data 15. The transformation data 15 may include spherical harmonic basis coefficients of two channels of R and G. Then, the data format of the transformation data 15 may be converted into the data storage format of the first device to obtain the distance data and the square of the distance data. It is clear that after the transformation data 5a1 and the transformation data 5a2 are obtained, the transformation data 5a1 may also be used as the R channel, and the transformation data 5a2 may be used as the G channel, and channel combination is performed on the transformation data 5a1 and the transformation data 5a2 to obtain the transformation data 5. The transformation data 5 may include spherical harmonic basis coefficients of two channels of R and G. Then, subsequent steps S804a, S805a and S806a (or S804b, S805b and S805b) are executed, which is not limited in this application.

[0344] In the following, the decoding procedure shown in FIG. 4c(2) is used as an example to explain in detail the process of obtaining probe data through decoding.

[0345] 9 is a diagram of an example of a decoding procedure. The decoding procedure may include the following steps: S901: Obtain a bitstream; S902: Perform entropy decoding; S903: Perform inverse quantization; S904: Perform inverse normalization; S905: Perform dimension conversion; S906: Perform format conversion.

[0346] S901: Obtain a bitstream, that is, obtain a bitstream.

[0347] For example, the bitstream may include intermediate data corresponding to illumination data for K probes and / or intermediate data corresponding to visibility data for K probes.

[0348] S902: Perform entropy decoding, that is, perform entropy decoding on the bitstream obtained in S901 to obtain intermediate data corresponding to the probe data.

[0349] For example, for S902, please refer to the description of S502, and the details will not be described again here.

[0350] In the embodiment in Fig. 9, the intermediate data is a two-dimensional matrix. It should be understood that the intermediate data may alternatively be a matrix of other dimensions (e.g., a five-dimensional matrix). This is not limited in this application. In the following, an explanation is provided by using an example in which the intermediate data is a two-dimensional matrix.

[0351] For example, the intermediate data corresponding to the illumination data of the K probes may include a two-dimensional matrix 11 whose elements are represented by eigenvector coefficients and a two-dimensional matrix 12 whose elements are represented by eigenvectors. Each element in the two-dimensional matrix 11 may be an eigenvector coefficient, and each element in the two-dimensional matrix 12 may be an eigenvector.

[0352] For example, the intermediate data corresponding to the visibility data of the K probes may be a two-dimensional matrix 21 whose elements are represented by eigenvector coefficients and a two-dimensional matrix 22 whose elements are represented by eigenvectors. Each element in the two-dimensional matrix 21 may be an eigenvector coefficient, and each element in the two-dimensional matrix 22 may be an eigenvector.

[0353] S903: Perform inverse quantization, that is, inverse quantize the intermediate data obtained in S902 to obtain transformed data 1.

[0354] For example, for S903, please refer to the description of S604, and the details will not be described again here.

[0355] For example, when the intermediate data is intermediate data corresponding to illumination data, the eigenvector coefficients in two-dimensional matrix 11 may be dequantized to obtain two-dimensional matrix 13, and the eigenvectors in two-dimensional matrix 12 may be dequantized to obtain two-dimensional matrix 14. Thus, the obtained transformation data 1 includes two-dimensional matrix 13 whose elements are represented by eigenvector coefficients and two-dimensional matrix 14 whose elements are represented by eigenvectors. Each element in two-dimensional matrix 13 is a dequantized eigenvector coefficient, and each element in two-dimensional matrix 14 is a dequantized eigenvector.

[0356] For example, when the intermediate data is intermediate data corresponding to visibility data, the eigenvector coefficients in two-dimensional matrix 21 may be dequantized to obtain two-dimensional matrix 23, and the eigenvectors in two-dimensional matrix 22 may be dequantized to obtain two-dimensional matrix 24. Thus, the obtained transformation data 1 includes two-dimensional matrix 23 whose elements are represented by eigenvector coefficients and two-dimensional matrix 24 whose elements are represented by eigenvectors. Each element in two-dimensional matrix 23 is a dequantized eigenvector coefficient, and each element in two-dimensional matrix 24 is a dequantized eigenvector.

[0357] S904: Perform inverse normalization, that is, perform inverse normalization on the transformed data 1 obtained in S903 to obtain transformed data 13.

[0358] For example, for S904, please refer to the description of S604, and the details will not be described again here.

[0359] For example, when the intermediate data is intermediate data corresponding to illumination data, denormalization may be performed on the eigenvector coefficients in two-dimensional matrix 13 to obtain two-dimensional matrix 15, and denormalization may be performed on the eigenvectors in two-dimensional matrix 14 to obtain two-dimensional matrix 26. Thus, the obtained transformation data 13 includes two-dimensional matrix 15 whose elements are represented by eigenvector coefficients and two-dimensional matrix 16 whose elements are represented by eigenvectors. Each element in two-dimensional matrix 15 is a dequantized eigenvector coefficient, and each element in two-dimensional matrix 16 is a dequantized eigenvector.

[0360] For example, when the intermediate data is intermediate data corresponding to visibility data, denormalization may be performed on the eigenvector coefficients in two-dimensional matrix 23 to obtain two-dimensional matrix 25, and denormalization may be performed on the eigenvectors in two-dimensional matrix 24 to obtain two-dimensional matrix 26. In this way, the obtained transformation data 13 includes two-dimensional matrix 25 whose elements are represented by eigenvector coefficients and two-dimensional matrix 26 whose elements are represented by eigenvectors. Each element in two-dimensional matrix 25 is an eigenvector coefficient obtained through denormalization, and each element in two-dimensional matrix 26 is an inverse quantized eigenvector.

[0361] It should be noted that in the rendering process, the first device may determine the shading effect of an object in a three-dimensional scene based on the eigenvectors and corresponding eigenvector coefficients without converting the eigenvector coefficients into spherical harmonic basis coefficients, spherical wavelet basis coefficients, or a two-dimensional picture.

[0362] S905: Perform dimension transformation, that is, perform dimension transformation on the transformed data 13 obtained in S904 to obtain transformed data 17.

[0363] For example, when the intermediate data is intermediate data corresponding to illumination data, a dimensional transformation may be performed on the two-dimensional matrix 15, and an eigenvector coefficient corresponding to the illumination data of each probe is extracted from the two-dimensional matrix 15, and a dimensional transformation is performed on the two-dimensional matrix 16, and an eigenvector corresponding to the illumination data of each probe is extracted from the two-dimensional matrix 16. Then, the eigenvectors and the eigenvector coefficients corresponding to the illumination data of the probes are arranged together.

[0364] For example, when the intermediate data is intermediate data corresponding to visibility data, a dimensional transformation may be performed on two-dimensional matrix 25, and eigenvector coefficients corresponding to the visibility data of each probe are extracted from two-dimensional matrix 25, and a dimensional transformation is performed on two-dimensional matrix 26, and eigenvectors corresponding to the illumination data of each probe are extracted from two-dimensional matrix 26. Then, the eigenvectors and eigenvector coefficients corresponding to the visibility data of the probes are placed together.

[0365] For example, the data obtained by performing a dimensional transformation on the transformed data 13 may be called transformed data 17, and the transformed data 17 may also be a matrix. It should be noted that the transformed data 17 is an intermediate result generated in the process of rearrangement of the first scheme in Fig. 4c(2).

[0366] For example, the number of elements required by the transformed data 17 may be determined, and a dimensional transformation is performed on the transformed data 13 based on the number of elements required by the transformed data 17 and the number of elements in the transformed data 13. For example, when it is determined that the number of elements required by the transformed data 17 is greater than the number of elements in the transformed data 13, the number of invalid elements may be determined based on the number of elements required by the transformed data 17 and the number of elements in the transformed data 13, and invalid elements having a corresponding number are discarded in the process of performing the dimensional transformation on the transformed data 13.

[0367] For example, if the size of the transformed data 17 is 150*240, it may be determined that the number of elements required by the transformed data 17 is 36000. If the size of the transformed data 13 is 190*190, it may be determined that the number of elements in the transformed data 13 is 36100. In this case, in the process of performing a dimensional transformation on the transformed data 13, 100 invalid elements may be discarded.

[0368] S906: Perform format conversion, ie, convert the data format of the converted data 17 obtained in S905 into the data storage format of the first device to obtain the probe data.

[0369] For example, a format conversion may be performed on the eigenvector coefficients and eigenvectors corresponding to the illumination data in the conversion data 17 to convert the data format of the eigenvector coefficients and eigenvectors corresponding to the illumination data into the data storage format of the first device. In this manner, the eigenvector coefficients and eigenvectors obtained through the format conversion and corresponding to the illumination data may form the illumination data.

[0370] For example, a format conversion may be performed on the eigenvector coefficients and eigenvectors corresponding to the visibility data in the transformation data 17 to convert the data format of the eigenvector coefficients and eigenvectors corresponding to the visibility data to the data storage format of the first device. In this manner, the eigenvector coefficients and eigenvectors obtained through the format conversion and corresponding to the visibility data may form the visibility data.

[0371] FIG. 10 is a diagram of an example of an electronic device. In the embodiment in FIG. 10, the electronic device is a first device, which may be configured to perform the method of the above embodiment. Therefore, for the beneficial effects that can be achieved by the first device, please refer to the beneficial effects in the corresponding method provided above. Details will not be described again here. The first device is: a bitstream acquisition module 1001 configured to acquire a bitstream, the bitstream including intermediate data encoded by a second device; a decoding module 1002 configured to decode the bitstream to obtain intermediate data; a data format conversion module 1003 configured to perform data format conversion on the intermediate data to obtain probe data, the probe data corresponding to one or more probes in the three-dimensional scene, the probe data being for determining shading effects of objects in the three-dimensional scene in a rendering process, and the data format conversion including a domain conversion; may include:

[0372] For example, the data format conversion module 1003 a domain transformation module 10031 configured to perform a first operation on the intermediate data to obtain transformed data, the first operation being domain transformation, or a domain transformation module 10031 configured to perform a second operation on the transformed data to obtain probe data, the second operation being domain transformation; an inverse quantization module 10032 configured to perform a first operation on the intermediate data to obtain transformed data, the first operation being inverse quantization; or an inverse quantization module 10032 configured to perform a second operation on the transformed data to obtain probe data, the second operation being inverse quantization; a relocation module 10033 configured to perform a first operation on the intermediate data to obtain transformed data, the first operation being a relocation of a first type; or a relocation module 10033 configured to perform a second operation on the transformed data to obtain probe data, the second operation being a relocation of the first type; Includes.

[0373] For example, the domain transformation module 10031 is further configured to perform a third process on the transformed data obtained through the second process after the second process is performed on the transformed data and before the probe data is obtained, the third process being a domain transformation.

[0374] The inverse quantization module 10032 is further configured to perform a third operation on the transformed data obtained through the second operation after the second operation is performed on the transformed data and before the probe data is obtained, the third operation being inverse quantization.

[0375] The rearrangement module 10033 is further configured to perform a third process on the transformed data acquired through the second process after the second process is performed on the transformed data and before the probe data is acquired, the third process being a rearrangement of the first type.

[0376] For example, the intermediate data is data on a YUV plane, and the rearrangement module is further configured to perform a second type of rearrangement on the intermediate data before the first processing is performed on the intermediate data, the second type of rearrangement including extracting data from the YUV plane.

[0377] For example, the second manner of rearrangement further includes arranging the data extracted from the YUV plane into a two-dimensional picture.

[0378] For example, when the intermediate data is intermediate data corresponding to lighting data, the first manner of rearrangement includes at least one of the following: adding a channel to the intermediate data corresponding to the lighting data, converting the intermediate data corresponding to the lighting data into a data storage format of the first device, or performing a dimensional transformation on the intermediate data corresponding to the lighting data.

[0379] For example, when the intermediate data is intermediate data corresponding to visibility data and the intermediate data corresponding to the visibility data includes a group of multiple channels, the first manner of rearrangement includes at least one of the following: combining the group of multiple channels, converting the intermediate data corresponding to the visibility data into a data storage format of the first device, or performing a dimensional transformation on the intermediate data corresponding to the visibility data.

[0380] For example, the data format conversion module 1003 is specifically configured to perform data format conversion on the intermediate data based on the first attribute data obtained by decoding the bitstream to obtain the probe data.

[0381] For example, the domain transformation includes at least one of the following: a normalized domain to a non-normalized domain transformation, a non-linear domain to a linear domain transformation, a YUV domain to an RGB domain transformation, an XYZ domain to an RGB domain transformation, and a Lab domain to an RGB domain transformation.

[0382] For example, the bitstream further includes attribute data of the probe encoded by the second device, the attribute data including first attribute data for data format conversion and / or second attribute data used in the rendering process.

[0383] For example, when the intermediate data includes intermediate data corresponding to lighting data and intermediate data corresponding to visibility data, the bitstream further includes bitstream structure information, which includes the location of the intermediate data corresponding to the lighting data and / or the location of the intermediate data corresponding to the visibility data.

[0384] For example, the probe data includes lighting data and / or visibility data when the intermediate data includes intermediate data corresponding to lighting data and intermediate data corresponding to visibility data.

[0385] Figure 11a is an example diagram of the compression effect, where the ordinate is the amount of data in kb and the abscissa is frames.

[0386] See Fig. 11a. For example, the probe data before encoding is larger than 40000kb. After encoding is performed by using the encoding method corresponding to the decoding method in this application, the data amount corresponding to the bitstream of most frames received by the first device belongs to the interval of 8kb-100kb, and the data amount of the bitstream corresponding to some frames is about 2000kb. It can be seen that the encoding method corresponding to the decoding method in this application can greatly reduce the bit rate and reduce the amount of data transmitted from the encoder side to the decoder side (first device). Furthermore, when the first device decodes the bitstream by using the decoding method in this application, the rendering delay of the first device can be reduced.

[0387] Fig. 11b is a diagram of an example of compression effect. Fig. 11b shows a PSNR (Peak Signal-to-Noise Ratio) curve corresponding to lossless compression and a PSNR curve corresponding to the decoding method in this application. PSNR may indicate the difference between an unprocessed picture (a picture before compression performed by the encoder side in Fig. 11b) and a processed picture (a picture obtained by decoding the bitstream sent by the encoder side in Fig. 11b by the decoder side). A larger PSNR indicates a smaller loss of the processed picture compared to the unprocessed picture. PSNR may be for measuring the quality or rendering effect of a compressed or decompressed picture. In Fig. 11b, the ordinate is PSNR and the abscissa is the number of frames.

[0388] Refer to FIG. 11b. For example, the curve corresponding to reference (delay 1 frame) is a PSNR curve corresponding to lossless compression in the prior art when one frame is delayed, and the average PSNR of the lossless compression is 58.49 dB. The curve corresponding to probe compression (delay 1 frame) is a PSNR curve corresponding to lossy compression performed by using an encoding method corresponding to the decoding method in this application when one frame is delayed, and the average PSNR of the lossy compression is 50.59 dB. The curve corresponding to reference (delay 4 frame) is a PSNR curve corresponding to lossless compression in the prior art when four frames are delayed, and the average PSNR of the lossy compression is 51.15 dB. The curve corresponding to probe compression (delay 4 frame) is a PSNR curve corresponding to lossy compression performed by using an encoding method corresponding to the decoding method in this application when four frames are delayed, and the average PSNR of the lossy compression is 47.97 dB.

[0389] It should be noted that for two pictures with PSNR greater than 40 dB, the difference perceived by the user on the two pictures is not obvious. It can be seen that the encoding method in this application does not significantly reduce the rendering effect.

[0390] It should be noted that if lossy compression is performed after data format conversion in the prior art, the accuracy of the probe data obtained through decoding by the decoder side will be seriously affected, and then the rendering effect will be seriously affected. After data format conversion in the encoding method corresponding to the decoding method in this application is performed, both lossless compression and lossy compression may be performed on the processed intermediate data. As shown in FIG. 11b, even if lossy compression is performed on the intermediate data obtained through data format conversion in the encoding method corresponding to the decoding method in this application, the decoder side can still obtain a high quality picture, and the user can hardly perceive the difference between the rendering effects through the naked eye. However, in the encoding method corresponding to the decoding method in this application, after data format conversion and lossy compression are performed on the probe data, the amount of data can be greatly reduced, the bit rate can be greatly reduced, and the data to be transmitted can be reduced, thereby reducing the rendering delay of the client and improving the user experience, especially in scenes with high requirements for real-time performance.

[0391] 12 shows an example of an apparatus 1200 according to an embodiment of the present application. The apparatus 1200 may include a processor 1201 and a communication interface 1202. Optionally, the apparatus 1200 further includes a memory 1203.

[0392] The components of the device 1200 are coupled together through a bus 1204. In addition to a data bus, the bus 1204 may further include a power bus, a control bus, and a status signal bus. However, for clarity of illustration, the various types of buses in the figures may be referred to as the bus 1204.

[0393] Optionally, the memory 1203 may be configured to store instructions in the above method embodiments. The processor 1201 may be configured to execute the instructions in the memory 1203 and control the communication interface to receive / transmit signals.

[0394] The apparatus 1200 may be an electronic device or a chip of an electronic device in the above method embodiments.

[0395] The processor 1201 is configured to process data, control data access and storage, issue commands, and control other components to execute operations. The processor 1201 may be implemented as one or more processors, one or more controllers, and / or other structures available for executing programs. The processor 1201 may specifically include at least one of a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic components. A general-purpose processor may include a microprocessor, and any conventional processor, controller, microcontroller, or state machine. The processor 1201 may alternatively be implemented as a combination of computing components, e.g., a DSP and a microprocessor.

[0396] The memory 1203 may include a computer-readable recording medium, such as a magnetic recording device (e.g., a hard disk, a floppy disk, and a magnetic stripe), an optical recording medium (e.g., a digital versatile disk (DVD)), a smart card, a flash memory device, a random access memory (RAM), a read only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), a register, and combinations thereof. The memory 1203 may be coupled to the processor 1201 such that the processor 1201 can read information and write information to the memory 1203. In particular, the memory 1203 may be integrated with the processor 1201, or the memory 1203 and the processor 1201 may be separate.

[0397] The communication interface 1202 may include circuitry and / or programming for implementing bidirectional communication between the terminal 1200 and one or more wireless network devices (e.g., routers, switches, and access points). The communication interface 1202 includes at least one receiving circuit and / or at least one transmitting circuit. In an embodiment, the communication interface 1202 may be implemented in part or in whole by a wireless modem.

[0398] All the relevant contents of the steps in the above method embodiments may be cited in the functional descriptions of the corresponding functional modules, and the details will not be described again here.

[0399] The embodiment further provides a computer-readable storage medium, which stores computer instructions, which, when executed on an electronic device, enable the electronic device to execute the steps of the above related method to realize the decoding method in the above embodiment.

[0400] The embodiment further provides a computer program product, which, when executed on a computer, enables the computer to execute the relevant steps above to realize the decoding method in the above embodiment.

[0401] Furthermore, an embodiment of this application further provides an apparatus. The apparatus may specifically be a chip, a component or a module. The apparatus may include a processor and a memory connected together. The memory is configured to store computer-executable instructions. When the apparatus operates, the processor may execute the computer-executable instructions stored in the memory to enable the chip to execute the decoding method in the above method embodiment.

[0402] The electronic device, computer readable storage medium, computer program product or chip provided in the embodiments are configured to execute the corresponding method provided above. Therefore, for the beneficial effects that can be achieved, refer to the beneficial effects in the corresponding method provided above. Details are not described again here.

[0403] Based on the above description of the implementation manner, those skilled in the art may understand that for the purpose of convenient and concise description, the above division into functional modules is used only as an example for explanation. In practical application, the above functions can be allocated to different functional modules for implementation based on requirements, that is, the internal structure of the device is divided into different functional modules to realize all or part of the above functions.

[0404] In some embodiments provided in this application, it should be understood that the disclosed apparatus and method may be realized in other ways. For example, the described apparatus embodiments are merely examples. For example, the division into modules or units is merely a logical division of functions, and other divisions may be made in the actual implementation. For example, multiple units or components may be combined or integrated into other devices, or some features may be ignored or not implemented. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be realized through some interfaces. Indirect couplings or communication connections between apparatuses or units may be realized in electronic, mechanical, or other forms.

[0405] The units described as separate parts may or may not be physically separate, and the parts shown as units may be one or more physical units, located in one place, or distributed in different places. Some or all of the units may be selected based on actual requirements to achieve the objectives of the solutions of the embodiments.

[0406] Furthermore, the functional units in the embodiments of this application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The integrated unit may be realized in the form of hardware or in the form of a software functional unit.

[0407] Any content in the embodiments of this application, and any content in the same embodiment, can be freely combined. Any combination of the above content falls within the scope of this application.

[0408] When the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, the integrated unit may be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of this application may be essentially realized in the form of a software product, or the part contributing to the prior art or all or part of the technical solutions. The software product is stored in a storage medium and includes some instructions for instructing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in the embodiments of this application. The above storage medium includes any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0409] The above describes the embodiments of this application with reference to the accompanying drawings. However, this application is not limited to the above specific implementation manner. The above specific implementation manner is merely an example and is not limiting. Taking inspiration from this application, those skilled in the art may further make modifications without departing from the purpose of this application and the protection scope of the claims, and all modifications shall fall within the protection scope of this application.

[0410] The method or algorithm steps described in combination with the contents disclosed in the embodiments of this application may be realized by hardware or by a processor by executing software instructions. The software instructions may include corresponding software modules. The software modules may be stored in a Random Access Memory (RAM), a flash memory, a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a register, a hard disk, a removable hard disk, a Compact Disk Read-Only Memory (CD-ROM), or any other form of storage medium known in the art. For example, the storage medium is coupled to the processor to enable the processor to read information from the storage medium and write information to the storage medium. It is clear that the storage medium may be a component of the processor. The processor and the storage medium may be located in an ASIC.

[0411] Those skilled in the art should recognize that the functions described in the embodiments of this application in one or more of the above examples may be realized by hardware, software, firmware, or any combination thereof. When the functions are realized by software, the functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes in a computer-readable medium. Computer-readable media include computer-readable storage media and communication media, and communication media include any medium that allows a computer program to be transmitted from one place to another. Storage media may be any available medium accessible to a general-purpose computer or a special-purpose computer.

[0412] The above describes the embodiments of this application with reference to the accompanying drawings. However, this application is not limited to the above specific implementation manner. The above specific implementation manner is merely an example and is not limiting. Taking inspiration from this application, those skilled in the art may further make modifications without departing from the purpose of this application and the protection scope of the claims, and all modifications shall fall within the protection scope of this application.

Claims

1. A method of decoding applied to a first device, comprising: obtaining a bitstream, the bitstream including intermediate data encoded by a second device; decoding the bitstream to obtain the intermediate data; performing a data format conversion on the intermediate data to obtain probe data, the probe data corresponding to one or more probes in a three-dimensional scene, the probe data being for determining shading effects of objects in the three-dimensional scene in a rendering process; Including, The method, wherein the data format transformation includes a domain transformation.

2. The step of performing data format conversion on the intermediate data to obtain probe data includes: performing a first process on the intermediate data to obtain transformed data; performing a second process on the converted data to obtain the probe data; Including, When the first processing is the domain transformation, the second processing includes inverse quantization and / or rearrangement of the first scheme; The method of claim 1 , wherein when the second operation is the domain transformation, the first operation includes the inverse quantization and / or rearrangement of the first scheme.

3. The step of performing data format conversion on the intermediate data to obtain probe data includes:

3. The method of claim 2, further comprising the step of: performing a third process on the transformed data acquired through the second process after the second process is performed on the transformed data and before the probe data is acquired, the third process including at least one of the following: the domain transformation, the inverse quantization, or the first scheme rearrangement.

4. The intermediate data is data on a YUV plane, and before the step of performing a first process on the intermediate data, the method includes:

4. The method of claim 2, further comprising the step of performing a second manner of rearrangement on the intermediate data, the second manner of rearrangement comprising extracting data from the YUV plane.

5. The method of claim 4 , wherein the second manner of rearrangement further comprises arranging the data extracted from the YUV plane into a two-dimensional picture.

6. 6. The method according to claim 2, wherein when the intermediate data is intermediate data corresponding to lighting data, the first manner of rearrangement comprises at least one of the following: adding a channel to the intermediate data corresponding to the lighting data, converting the intermediate data corresponding to the lighting data into a data storage format of the first device, or performing a dimensional transformation on the intermediate data corresponding to the lighting data.

7. 6. The method of claim 2, wherein when the intermediate data is intermediate data corresponding to visibility data and the intermediate data corresponding to the visibility data includes a group of multiple channels, the first manner of rearrangement includes at least one of the following: combining the groups of multiple channels, converting the intermediate data corresponding to the visibility data into a data storage format of the first device, or performing a dimensional transformation on the intermediate data corresponding to the visibility data.

8. The step of performing data format conversion on the intermediate data to obtain probe data includes: The method of claim 1 , further comprising: performing a data format conversion on the intermediate data based on first attribute data obtained by decoding the bitstream to obtain the probe data.

9. The method of claim 8 , wherein the first attribute data includes at least one of the following: a quantization parameter, a domain transformation parameter, and a relocation parameter.

10. 10. The method of claim 1, wherein the domain transformation comprises at least one of the following: normalized to non-normalized domain transformation, non-linear to linear domain transformation, YUV to RGB domain transformation, XYZ to RGB domain transformation, and Lab to RGB domain transformation.

11. 11. The method of claim 1, wherein the bitstream further comprises attribute data of the probe encoded by the second device, the attribute data comprising first attribute data for data format conversion and / or second attribute data used in the rendering process.

12. 12. A method according to any one of claims 1 to 11, when the intermediate data comprises intermediate data corresponding to lighting data and intermediate data corresponding to visibility data, the bitstream further comprising bitstream structure information, the bitstream structure information comprising a position of the intermediate data corresponding to the lighting data and / or a position of the intermediate data corresponding to the visibility data.

13. 13. The method of claim 1, wherein the probe data comprises the lighting data and / or the visibility data, when the intermediate data comprises the intermediate data corresponding to the lighting data and the intermediate data corresponding to the visibility data.

14. A first device, a bitstream acquisition module configured to acquire a bitstream, the bitstream including intermediate data encoded by a second device; a decoding module configured to decode the bitstream to obtain the intermediate data; a data format conversion module configured to perform a data format conversion on the intermediate data to obtain probe data, the probe data corresponding to one or more probes in a three-dimensional scene, the probe data being for determining shading effects of objects in the three-dimensional scene in a rendering process, the data format conversion including a domain conversion; A first device including

15. The data format conversion module includes: a domain transformation module configured to perform a first operation on the intermediate data to obtain transformed data, the first operation being the domain transformation, or a domain transformation module configured to perform a second operation on the transformed data to obtain the probe data, the second operation being the domain transformation; an inverse quantization module configured to perform a first operation on the intermediate data to obtain transformed data, the first operation being inverse quantization; or an inverse quantization module configured to perform a second operation on the transformed data to obtain the probe data, the second operation being inverse quantization; a relocation module configured to perform a first operation on the intermediate data to obtain transformed data, the first operation being a relocation of a first type; or a relocation module configured to perform a second operation on the transformed data to obtain the probe data, the second operation being a relocation of a first type; The first device of claim 14 , comprising:

16. the domain transformation module is further configured to perform a third operation on the transformed data acquired through the second operation after the second operation is performed on the transformed data and before the probe data is acquired, the third operation being the domain transformation; the inverse quantization module is further configured to perform a third operation on the transformed data obtained through the second operation after the second operation is performed on the transformed data and before the probe data is obtained, the third operation being the inverse quantization; The first device of claim 15, wherein the relocation module is further configured to perform a third process on the conversion data acquired through the second process after the second process is performed on the conversion data and before the probe data is acquired, the third process being a relocation of the first type.

17. The intermediate data is data on a YUV plane, 17. The first device of claim 15 or 16, wherein the rearrangement module is further configured to perform a second type of rearrangement on the intermediate data before a first processing is performed on the intermediate data, the second type of rearrangement including extracting data from the YUV plane.

18. 20. The first device of claim 17, wherein the second manner of rearrangement further comprises arranging the data extracted from the YUV plane into a two-dimensional picture.

19. 18. The first device according to claim 15, wherein when the intermediate data is intermediate data corresponding to lighting data, the first manner of rearrangement comprises at least one of the following: adding a channel to the intermediate data corresponding to the lighting data, converting the intermediate data corresponding to the lighting data into a data storage format of the first device, or performing a dimensional transformation on the intermediate data corresponding to the lighting data.

20. 18. The first device of claim 15, wherein when the intermediate data corresponds to visibility data and the intermediate data corresponding to the visibility data includes a group of multiple channels, the first manner of rearrangement includes at least one of the following: combining the group of multiple channels, converting the intermediate data corresponding to the visibility data into a data storage format of the first device, or performing a dimensional transformation on the intermediate data corresponding to the visibility data.

21. The first device of claim 14, wherein the data format conversion module is specifically configured to perform data format conversion on the intermediate data based on first attribute data obtained by decoding the bitstream to obtain the probe data.

22. The first device of claim 21 , wherein the first attribute data includes at least one of the following: a quantization parameter, a domain transformation parameter, and a rearrangement parameter.

23. 23. The first device of claim 14, wherein the domain transformation includes at least one of the following: a normalized domain to a non-normalized domain transformation, a non-linear domain to a linear domain transformation, a YUV domain to an RGB domain transformation, an XYZ domain to an RGB domain transformation, and a Lab domain to an RGB domain transformation.

24. 24. A first device according to any one of claims 14 to 23, wherein the bitstream further comprises attribute data of the probe encoded by the second device, the attribute data comprising the first attribute data for data format conversion and / or second attribute data used in the rendering process.

25. A first device according to any one of claims 14 to 24, when the intermediate data includes the intermediate data corresponding to the lighting data and the intermediate data corresponding to the visibility data, the bitstream further includes bitstream structure information, the bitstream structure information including a position of the intermediate data corresponding to the lighting data and / or a position of the intermediate data corresponding to the visibility data.

26. 26. A first device according to any one of claims 14 to 25, wherein when the intermediate data includes intermediate data corresponding to the illumination data and intermediate data corresponding to the visibility data, the probe data includes the illumination data and / or the visibility data.

27. An electronic device including a memory and a processor, the memory is coupled to the processor; An electronic device, the memory storing program instructions which, when executed by the processor, enable the electronic device to execute a decoding method according to any one of claims 1 to 13 executed by any first device.

28. A chip including one or more interface circuits and one or more processors, A chip, wherein the interface circuit is configured to receive a signal from a memory of an electronic device and transmit the signal to the processor, the signal including computer instructions, which when executed by the processor enables the electronic device to perform the decoding method of any one of claims 1 to 13 executed by any first electronic device.

29. 1. A computer-readable storage medium, comprising: The computer-readable storage medium stores a computer program which, when executed on a computer or a processor, enables the computer or the processor to execute the decoding method according to any one of claims 1 to 13.

30. 1. A computer program product comprising: The computer program product comprises a software program, which, when executed by a computer or processor, performs the steps of the method according to any one of claims 1 to 13.

31. 1. A computer-readable storage medium, comprising: A computer readable storage medium storing a bit stream, and probe data is obtained when the bit stream is decoded by using a decoding method according to any one of claims 1 to 13.

Images