Encoding method and electronic device

Abstract

An embodiment of the present application provides an encoding method and an electronic device. The method includes: obtaining probe data corresponding to one or more probes in a three-dimensional scene and used by a second device to determine shading effects of objects in the three-dimensional scene in a rendering process; performing data format conversion, including domain conversion, on the probe data to obtain intermediate data; and encoding the intermediate data to obtain a corresponding bitstream. Compared with the prior art, the present application can reduce the bitrate under the same rendering effect, or improve the rendering effect under the same bitrate.

Description

[Technical Field]

[0001] TECHNICAL FIELD Embodiments of the present application relate to the field of encoding and decoding, and in particular to encoding methods and electronic devices. [Background technology]

[0002] As people's demands for the quality of rendered images gradually increase, in order to make images clearer, the means of simulating the shading effect in the rendering process also gradually shift from simulating the shading effect of direct lighting (i.e., simulating the shading effect caused by a single reflection of a light ray) to simulating the shading effect of indirect lighting (i.e., simulating the shading effect caused by multiple reflections of a light ray). Probes are one of the means for simulating the shading effect of indirect lighting.

[0003] Currently, in the device-cloud synergy scenario, the cloud generates probe data, compresses it, and sends it to the device. After receiving the bitstream, the device decompresses the bitstream to obtain the probe data, and then uses the decoded probe data to perform the rendering process, calculating the indirect shading effect generated by light rays reflected by objects in a 3D (three-dimensional) scene. Summary of the Invention

[0004] The present application provides an encoding method and an electronic device, which, compared with the prior art, can reduce the bit rate under the same rendering effect, or can improve the rendering effect under the same bit rate.

[0005] According to a first aspect, an embodiment of the present application comprises: , applied to the first deviceAn encoding method is provided. The method includes: acquiring probe data corresponding to one or more probes in a three-dimensional scene, the probe data being used by a second device to determine shading effects of objects in the three-dimensional scene in a rendering process, the objects in the three-dimensional scene corresponding to three-dimensional models in the three-dimensional scene, the models may include object models or human models; performing data format conversion on the probe data to obtain intermediate data, the data format conversion including domain conversion; and encoding the intermediate data to obtain a corresponding bitstream. In this way, the probe data is converted into a more compact representation by performing data format conversion on the probe data. Compared with the prior art, the bitrate can be reduced under the same rendering effect. Alternatively, the data format conversion is performed on the probe data to increase the number of bits in the bitrate occupied by data that is more important in the rendering process. Compared with the prior art, the rendering effect is improved under the same bitrate.

[0006] For example, the encoding method of the present application may be applied to N-end synergy scenes (N is an integer greater than 1), such as cloud gaming, cloud exhibition, interior decoration, clothing design, and architectural design scenes, which are not limited in the present application. The first device may be a server or a terminal. The second device may be a terminal.

[0007] For example, the encoding method of the present application is applied to a device-cloud synergy scenario, where the first device is a server and the second device is a terminal, such as a personal computer, a mobile phone, or a VR (virtual reality) wearable device.

[0008] For example, domain transformation may refer to converting the representation of data from one domain to another. Domains can be classified from different perspectives based on requirements. Examples are as follows:

[0009] From a normalization perspective, domains can be classified into normalized and non-normalized regions.

[0010] From a color space point of view, the domain can be classified into transformations into the RGB domain, the YUV domain, the XYZ domain, and the Lab domain.

[0011] From the point of view of numerical relations, the domains can be classified into linear and nonlinear domains. The nonlinear domains can be exponential domain, PQ (perceptual quantizer) domain, HLG (hybrid log gamma) domain, etc.

[0012] From the perspective of numerical representation format, the domain can be classified into a picture domain and a transform domain. For example, the picture domain may be a domain represented by a picture. For example, the transform domain may be a domain represented by transform basis functions and corresponding transform coefficients. For data Y(t) in the transform domain, an approximation can be performed on the data Y(t) by using x transform bases e_1(t) to e_x(t), so that the data Y(t) is approximately equal to the sum of the products of the x transform bases and the corresponding transform coefficients. The transform bases include, but are not limited to, spherical harmonic bases, spherical wavelet bases, eigenvectors, etc. This is not limited in this application.

[0013] For example, the terms RGB domain, YUV domain, XYZ domain, and Lab domain refer to the CIE 1931 RGB color space, the YUV color space (including variables such as YCbCr, YPbPr, and YCoCg), the CIE 1931 XYZ color space, and the CIELAB color space, respectively.

[0014] For example, any video encoding scheme may be used to encode the intermediate data to obtain a bitstream corresponding to the probe data. For example, HEVC (high efficiency video coding) encoding may be performed on the intermediate data to obtain a corresponding bitstream. As another example, AVC (advanced video coding) encoding may be performed on the intermediate data to obtain a corresponding bitstream. As yet another example, VVC (versatile video coding) encoding may be performed on the intermediate data to obtain a corresponding bitstream. As yet another example, entropy encoding may be performed on the intermediate data to obtain a corresponding bitstream. As yet another example, the intermediate data may alternatively be encoded with another video encoding scheme, such as AVS (audio video coding standard), to obtain a corresponding bitstream. This is not a limitation of the present application.

[0015] According to the first aspect, performing data format conversion on the probe data to obtain intermediate data includes performing a first operation on the probe data to obtain transformed data and performing a second operation on the transformed data to obtain intermediate data. If the first operation is a domain conversion, the second operation includes quantization and / or first modality rearrangement, and if the second operation is a domain conversion, the first operation includes quantization and / or first modality rearrangement. Thus, data format conversion may be performed on the probe data by combining domain conversion and quantization, by combining domain conversion and first modality rearrangement, or by combining domain conversion, quantization, and first modality rearrangement. Both quantization and first modality rearrangement can convert data into a more compact representation format, and then data format conversion is performed on the probe data by combining domain conversion, quantization, and / or first modality rearrangement, thereby further reducing the bit rate.

[0016] When a combination of domain transformation, quantization, and / or first modality rearrangement is used, the order in which the domain transformation, quantization, and first modality rearrangement are performed is not limited herein, nor is the number of times any one of the domain transformation, quantization, and first modality rearrangement processing methods is performed.

[0017] For example, quantization may include uniform quantization or non-uniform quantization. Non-uniform quantization includes, but is not limited to, μ-law quantization, a-law quantization, etc. This is not a limitation of the present application. Furthermore, quantization may further include adaptive quantization and other types of quantization. This is also not a limitation of the present application.

[0018] For example, the transformed data of the present application may include at least one of transformed data 1 to transformed data 15 in the following embodiments.

[0019] According to the first aspect or any one of the above implementations of the first aspect, performing data format conversion on the probe data to obtain intermediate data further includes performing a third process on the probe data before performing the first process on the probe data. The third process includes at least one of the following: domain conversion, quantization, and first modality rearrangement. In this way, the number of times data format conversion is performed on the probe data can be increased, thereby further reducing the bit rate.

[0020] According to the first aspect or any one of the above implementations of the first aspect, after the second processing is performed on the converted data and before the intermediate data is obtained, the method further includes performing a second format rearrangement on the data obtained by the second processing to obtain the intermediate data. The second format rearrangement is padding the data obtained by the second processing into a YUV plane. In this way, compression is advantageous and the bit rate can be further reduced.

[0021] According to the first aspect or any one of the above implementations of the first aspect, when the probe data is illuminance data, and when the illuminance data includes multiple channels, the first modality rearrangement includes at least one of the following: discarding data of some channels, converting the illuminance data to a preset precision, or performing a dimensional transformation on the illuminance data.

[0022] For example, the preset precision may be set to be equal to or greater than a requirement-based precision threshold to ensure the precision of the illuminance data sent to the second device and ensure a good rendering effect. The precision threshold may be set based on requirements. For example, if the precision threshold is a 16-bit floating-point number, the preset precision may be set to a 16-bit floating-point number or a 32-bit floating-point number. This is not limited to this specification. Furthermore, since the precision of illuminance data generated in different application scenarios may be different, the illuminance data in different application scenarios may be converted to the same precision by converting the data to the preset precision, thereby facilitating subsequent data format conversion.

[0023] It should be noted that the "illuminance data" in "converting the illuminance data to a preset precision" may be the original illuminance data, the illuminance data obtained by domain transformation, the illuminance data obtained by quantization, the illuminance data obtained by discarding data of some channels, or the illuminance data obtained by dimension transformation.

[0024] It should be noted that the "illuminance data" in "performing a dimensional transformation on the illuminance data" may be the original illuminance data, the illuminance data obtained by a dimensional transformation, the illuminance data obtained by quantization, the illuminance data obtained by discarding data of some channels, or the illuminance data converted to a preset precision.

[0025] The discarded data of a channel may be data that is less relevant to the second device's determination of the shading effect of an object in a three-dimensional scene during the rendering process. Therefore, discarding data of some channels can improve the encoding speed and further reduce the delay in transmitting probe data from the first device to the second device.

[0026] For example, illuminance data can be used to describe the radiant illumination of objects around the probe.

[0027] For example, the probe data may be a matrix. For example, dimensional transformation may refer to changing the matrix size. For example, if the probe data is a 100x1000x3 matrix, the probe data may be transformed into a 50x60x100 matrix through dimensional transformation. As another example, if the probe data is a 150x240x3 matrix, the probe data may be transformed into a 190x190x3 matrix through dimensional transformation, with the missing portion being filled in by using invalid data.

[0028] According to the first aspect or any one of the above implementations of the first aspect, when the probe data is visibility data, and when the visibility data includes multiple channels, the first modality rearrangement includes at least one of the following: performing channel splitting, converting the visibility data to a preset precision, or performing a dimensional transformation on the visibility data.

[0029] It should be noted that the "visibility data" in "converting visibility data to a preset precision" may be original visibility data, visibility data obtained by domain transformation, visibility data obtained by quantization, visibility data obtained by channel splitting, or visibility data obtained by dimension transformation.

[0030] It should be noted that the "visibility data" in "performing a dimensional transformation on the visibility data" may be the original visibility data, visibility data obtained by dimensional transformation, visibility data obtained by quantization, visibility data obtained by channel splitting, or visibility data converted to a preset precision.

[0031] For example, visibility data can be separated into multiple independent channels by channel splitting, and since the similarity between data of the same channel in different frames is higher than the similarity between data of different channels, it is easier to subsequently encode the data of the channels separately, thereby further reducing the bit rate.

[0032] For example, visibility data may be used to describe the distribution of distances between the probe and objects surrounding the probe (which may also be referred to as a depth distribution), including, but not limited to, distance data, variance of distance data, squared distance data, etc. This is not a limitation of the present application.

[0033] It should be noted that if the visibility data is single-channel data, the first modality rearrangement may include converting the visibility data to a preset precision and / or performing a dimensional transformation on the visibility data.

[0034] For example, the preset precision may be set to be equal to or greater than a precision threshold based on requirements to ensure the precision of the visibility data sent to the second device and ensure the rendering effect. The precision threshold may be set based on requirements. For example, if the precision threshold is a 16-bit floating-point number, the preset precision may be set to a 16-bit floating-point number or a 32-bit floating-point number. This is not limited in the present application. Furthermore, since the precision of visibility data generated in different application scenarios may be different, the visibility data in different application scenarios may be converted to the same precision by converting the data to the preset precision, thereby facilitating subsequent data format conversion.

[0035] In accordance with the first aspect or any one of the above implementations of the first aspect, the domain transformation includes at least one of the following: a transformation from a non-normalized domain to a normalized domain, a transformation from a linear domain to a non-linear domain, a transformation from an RGB domain to a YUV domain, a transformation from an RGB domain to an XYZ domain, a transformation from an RGB domain to a Lab domain, and a transformation from a picture domain to a transform domain.

[0036] According to the first aspect or any one of the above implementations of the first aspect, the probe data is represented by a two-dimensional picture, spherical harmonic basis coefficients, or spherical wavelet basis coefficients.

[0037] According to the first aspect or any one of the above implementations of the first aspect, the method further includes encoding attribute data of the one or more probes to obtain a bitstream, the attribute data including first attribute data of a data format conversion and / or second attribute data used in a rendering process.

[0038] For example, a third-modality rearrangement may first be performed on the attribute data of the probe, and then the attribute data obtained by the rearrangement is encoded. The third-modality rearrangement may be concatenated.

[0039] For example, the first attribute data includes at least one of the following: a quantization parameter, a domain transformation parameter, or a rearrangement parameter.

[0040] For example, the domain transformation parameters may include at least one of the following, which is not limited in this application: a normalization parameter, an exponential transformation parameter, a PQ transformation parameter, an HLG transformation parameter, and a color space transformation parameter.

[0041] According to the first aspect or any one of the above implementations of the first aspect, if the probe data includes illuminance data and visibility data, the bitstream includes bitstream structure information, and the bitstream structure information data includes the location of intermediate data corresponding to the illuminance data and / or the location of intermediate data corresponding to the visibility data.

[0042] For example, the bitstream structure information may further include, but is not limited to, the number of probes, the length and data format of intermediate data corresponding to illuminance data, the execution order of data format conversion types corresponding to illuminance data, the length and data format of intermediate data corresponding to visibility data, the execution order of data format conversion types corresponding to visibility data, the position, length and data format of first attribute data, the position, length and data format of second attribute data, etc. This is not limited in the present application. The data format conversion types may include various first-style rearrangement types (e.g., discarding data of some channels, channel splitting, precision conversion, and dimension conversion), various normalization types (e.g., adaptive normalization, fixed parameter normalization, min-max normalization, and z-score (standard score) normalization), various domain conversion types (e.g., conversion from a linear domain to a nonlinear domain, conversion from an RGB domain to a YUV domain, conversion from an RGB domain to an XYZ domain, conversion from an RGB domain to an Lab domain, and conversion from a picture domain to a transform domain), various quantization types (e.g., uniform quantization and non-uniform quantization), second-style rearrangement types such as padding data into YUV planes, and the like. This is not limited in the present application. It should be understood that the information that may be included in the bitstream structure information may be more or less than the information described above. This is not limited in the present application. Furthermore, the bitstream may alternatively not include bitstream structure information. This may be specifically set based on requirements. This is also not limited in the present application.

[0043] According to the first aspect or any one of the above implementations of the first aspect, if the probe data includes illuminance data and visibility data, the method further includes determining a first target bit rate corresponding to the illuminance data and a second target bit rate corresponding to the visibility data based on an amount of illuminance data, an amount of visibility data, and the predetermined bit rate. Encoding the intermediate data to obtain corresponding bit streams includes encoding the intermediate data corresponding to the illuminance data based on the first target bit rate and encoding the intermediate data corresponding to the visibility data based on the second target bit rate to obtain the bit rates.

[0044] For example, the ratio of the first target bit rate to the second target bit rate can be determined based on the amount of illuminance data and the amount of visibility data. In this way, appropriate bit rates can be allocated to the illuminance data and the visibility data. Compared with the case where the ratio of the target bit rate corresponding to the illuminance data to the target bit rate corresponding to the visibility data is fixed, in this application, the rendering effect can be improved when the bit rate is the same.

[0045] According to the first aspect or any one of the above implementations of the first aspect, if the probe data includes illuminance data and visibility data, the method further includes determining an encoding scheme corresponding to the illuminance data and an encoding scheme corresponding to the visibility data based on an amount of illuminance data, an amount of visibility data, and channel feedback information. The encoding scheme includes intra-frame encoding or inter-frame encoding. Encoding the intermediate data to obtain a corresponding bitstream includes encoding the intermediate data corresponding to the illuminance data by using the encoding scheme corresponding to the illuminance data, and encoding the intermediate data corresponding to the visibility data by using the encoding scheme corresponding to the visibility data to obtain a bitstream.

[0046] For example, intra-frame coding is a coding scheme in which only information about a current frame is used when the current frame is coded. For example, intra-frame coding of a probe data group can be completed by HEVC intra-frame coding. For example, inter-frame coding is a coding scheme in which information about a non-current frame is used when the current frame is coded. For example, inter-frame coding of a probe data group can be completed by HEVC inter-frame coding.

[0047] According to a second aspect, an embodiment of the present application provides a first device, the first device comprising: a data acquisition module configured to acquire probe data corresponding to one or more probes in the three-dimensional scene, the probe data being used by the second device to determine shading effects of objects in the three-dimensional scene in a rendering process; a data format conversion module configured to perform a data format conversion on the probe data to obtain intermediate data, the data format conversion including a domain conversion; an encoding module configured to encode the intermediate data to obtain a corresponding bitstream; Includes:

[0048] According to a second aspect, the data format conversion module: a domain transformation module configured to perform a first operation on the probe data to obtain transformed data, the first operation being a domain transformation, or to perform a second operation on the transformed data to obtain intermediate data, the second operation being a domain transformation; a quantization module configured to perform a first operation on the probe data to obtain transformed data, the first operation being quantization, or to perform a second operation on the transformed data to obtain intermediate data, the second operation being quantization; a rearrangement module configured to perform a first operation on the probe data to obtain transformed data, the first operation being a first modality rearrangement, or to perform a second operation on the transformed data to obtain intermediate data, the second operation being a first modality rearrangement; Includes:

[0049] According to the second aspect or any one of the above implementations of the second aspect, the domain transformation module is further configured to perform a third operation on the probe data before performing the first operation on the probe data, the third operation being a domain transformation.

[0050] The quantization module is further configured to perform a third operation on the probe data before performing the first operation on the probe data, the third operation being quantization.

[0051] The rearrangement module is further configured to perform a third operation on the probe data before performing the first operation on the probe data, the third operation being a first modality rearrangement.

[0052] According to the second aspect or any one of the above implementations of the second aspect, the rearrangement module is further configured, after performing the second processing on the transformed data and before obtaining the intermediate data, to perform a second-way rearrangement on the data obtained by the second processing to obtain the intermediate data, wherein the second-way rearrangement is padding the data obtained by the second processing into a YUV plane.

[0053] According to the second aspect or any one of the above implementations of the second aspect, when the probe data is illuminance data, and when the illuminance data includes multiple channels, the first modality rearrangement includes at least one of the following: discarding data of some channels, converting the illuminance data to a preset precision, or performing a dimensional transformation on the illuminance data.

[0054] According to the second aspect or any one of the above implementations of the second aspect, when the probe data is visibility data, and when the visibility data includes multiple channels, the first modality rearrangement includes at least one of the following: performing channel splitting, converting the visibility data to a preset precision, or performing a dimensional transformation on the visibility data.

[0055] In accordance with the second aspect or any one of the above implementations of the second aspect, the domain transformation includes at least one of the following: a transformation from a non-normalized domain to a normalized domain, a transformation from a linear domain to a non-linear domain, a transformation from an RGB domain to a YUV domain, a transformation from an RGB domain to an XYZ domain, a transformation from an RGB domain to a Lab domain, and a transformation from a picture domain to a transform domain.

[0056] According to the second aspect or any one of the above implementations of the second aspect, the probe data is represented by a two-dimensional picture, spherical harmonic basis coefficients, or spherical wavelet basis coefficients.

[0057] According to the second aspect or any one of the above implementations of the second aspect, the encoding module is further configured to encode attribute data of the one or more probes to obtain a bitstream, the attribute data including first attribute data of the data format conversion and / or second attribute data used in the rendering process.

[0058] According to the second aspect or any one of the above implementations of the second aspect, if the probe data includes illuminance data and visibility data, the bitstream includes bitstream structure information, and the bitstream structure information includes the location of intermediate data corresponding to the illuminance data and / or the location of intermediate data corresponding to the visibility data.

[0059] In accordance with the second aspect or any one of the above implementations of the second aspect, the first device comprises: When the probe data includes illuminance data and visibility data, the data processing device further includes a bitstream load balancing module configured to determine a first target bit rate corresponding to the illuminance data and a second target bit rate corresponding to the visibility data based on the amount of illuminance data, the amount of visibility data, and a preset bit rate.

[0060] The encoding module is particularly configured to encode the intermediate data corresponding to the illuminance data based on a first target bit rate, and encode the intermediate data corresponding to the visibility data based on a second target bit rate to obtain a bit rate.

[0061] In accordance with the second aspect or any one of the above implementations of the second aspect, the first device further includes:

[0062] When the probe data includes illuminance data and visibility data, the bitstream load balancing module is configured to determine an encoding scheme corresponding to the illuminance data and an encoding scheme corresponding to the visibility data based on the amount of illuminance data, the amount of visibility data, and channel feedback information, where the encoding scheme includes intra-frame encoding or inter-frame encoding.

[0063] The encoding module is particularly configured to encode the intermediate data corresponding to the illuminance data by using an encoding scheme corresponding to the illuminance data, and to encode the intermediate data corresponding to the visibility data by using an encoding scheme corresponding to the visibility data to obtain a bitstream.

[0064] The second aspect and any one of the implementations of the second aspect correspond to the first aspect and any one of the implementations of the first aspect, respectively. For technical effects corresponding to the second aspect and any one of the implementations of the second aspect, please refer to the technical effects corresponding to the first aspect and any one of the implementations of the first aspect. Details will not be described again here.

[0065] According to a third aspect, an embodiment of the present application provides an electronic device including a memory and a processor, the memory coupled to the processor, the memory storing program instructions, which, when executed by the processor, enable the electronic device to perform an encoding method according to the first aspect or any possible implementation of the first aspect.

[0066] The third aspect and any one of the implementations of the third aspect correspond to the first aspect and any one of the implementations of the first aspect, respectively. For technical effects corresponding to the third aspect and any one of the implementations of the third aspect, please refer to the technical effects corresponding to the first aspect and any one of the implementations of the first aspect. Details will not be described again here.

[0067] According to a fourth aspect, embodiments of the present application provide a chip including one or more interface circuits and one or more processors. The interface circuits are configured to receive signals from a memory of an electronic device and to transmit the signals to the processor, the signals including computer instructions. Execution of the computer instructions by the processor enables the electronic device to perform an encoding method according to the first aspect or any possible implementation of the first aspect.

[0068] The fourth aspect and any one of the implementations of the fourth aspect correspond to the first aspect and any one of the implementations of the first aspect, respectively. For technical effects corresponding to the fourth aspect and any one of the implementations of the fourth aspect, please refer to the technical effects corresponding to the first aspect and any one of the implementations of the first aspect. Details will not be described again here.

[0069] According to a fifth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program which, when executed by a computer or processor, enables the computer or processor to perform the encoding method according to the first aspect or any possible implementation of the first aspect.

[0070] The fifth aspect and any one of the implementations of the fifth aspect correspond to the first aspect and any one of the implementations of the first aspect, respectively. For technical effects corresponding to the fifth aspect and any one of the implementations of the fifth aspect, please refer to the technical effects corresponding to the first aspect and any one of the implementations of the first aspect. Details will not be described again here.

[0071] According to a sixth aspect, an embodiment of the present application provides a computer program product, comprising a software program that, when executed by a computer or a processor, enables the computer or processor to perform the encoding method according to the first aspect or any possible implementation of the first aspect.

[0072] The sixth aspect and any one of the implementations of the sixth aspect correspond to the first aspect and any one of the implementations of the first aspect, respectively. For technical effects corresponding to the sixth aspect and any one of the implementations of the sixth aspect, please refer to the technical effects corresponding to the first aspect and any one of the implementations of the first aspect. Details will not be described again here.

[0073] According to a seventh aspect, the present application provides a computer-readable storage medium storing a bitstream, the bitstream being obtained by using an encoding method according to the first aspect or any possible implementation of the first aspect.

[0074] The seventh aspect and any one of the implementations of the seventh aspect correspond to the first aspect and any one of the implementations of the first aspect, respectively. For technical effects corresponding to the seventh aspect and any one of the implementations of the seventh aspect, please refer to the technical effects corresponding to the first aspect and any one of the implementations of the first aspect. Details will not be described again here.

[0075] According to an eighth aspect, the present application provides a bitstream, the bitstream being obtained by using an encoding method according to the first aspect or any possible implementation of the first aspect. Optionally, the bitstream may be stored on a computer-readable storage medium or transmitted in the form of an electromagnetic signal.

[0076] The eighth aspect and any one of the implementations of the eighth aspect correspond to the first aspect and any one of the implementations of the first aspect, respectively. For technical effects corresponding to the eighth aspect and any one of the implementations of the eighth aspect, please refer to the technical effects corresponding to the first aspect and any one of the implementations of the first aspect. Details will not be described again here. [Brief explanation of the drawings]

[0077] [Figure 1a] FIG. 1 is a diagram of an example system framework. [Figure 1b] 1 is a diagram of an example of a probe distribution in a three-dimensional scene. [Figure 2a] FIG. 1 is a diagram of an example encoding framework. [Figure 2b] FIG. 10 is a diagram illustrating an example of the structure of a data format conversion module. [Figure 3a] FIG. 1 is a diagram of an example encoding procedure. [Figure 3b] FIG. 1 is a diagram of an example encoding procedure. [Figure 4a] FIG. 1 is a diagram of an example encoding procedure. [Figure 4b] FIG. 1 is a diagram of an example encoding procedure. [Figure 4c(1)] FIG. 1 is a diagram of an example encoding procedure. [Figure 4c(2)] FIG. 1 is a diagram of an example encoding procedure. [Figure 4c(3)] FIG. 1 is a diagram of an example encoding procedure. [Figure 4c(4)] FIG. 1 is a diagram of an example encoding procedure. [Figure 4c(5)] FIG. 1 is a diagram of an example encoding procedure. [Figure 4c(6)]FIG. 1 is a diagram of an example encoding procedure. [Figure 4c(7)] FIG. 1 is a diagram of an example encoding procedure. [Figure 5] FIG. 1 is a diagram of an example encoding procedure. [Figure 6] FIG. 1 is a diagram of an example encoding procedure. [Figure 7] FIG. 1 is a diagram of an example encoding procedure. [Figure 8] FIG. 1 is a diagram of an example encoding procedure. [Figure 9] FIG. 1 is a diagram of an example encoding procedure. [Figure 10] FIG. 1 is a diagram of an example encoding procedure. [Figure 11] FIG. 1 is a diagram of an example encoding procedure. [Figure 12] FIG. 1 is a diagram of an example of the structure of a bitstream. [Figure 13] FIG. 1 is a diagram of an example electronic device. [Figure 14a] FIG. 10 is a diagram of an example of the compression effect. [Figure 14b] FIG. 10 is a diagram of an example of the compression effect. [Figure 15] 1 is a diagram of an example of the structure of the device. DETAILED DESCRIPTION OF THE INVENTION

[0078] The following clearly and completely describes the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. It is clear that the described embodiments are only a part of the embodiments of the present application, and are not all of them. All other embodiments that a person skilled in the art can obtain based on the embodiments of the present application without creative efforts should fall within the protection scope of the present application.

[0079] The term "and / or" in this specification refers only to an association relationship describing related objects and indicates that three relationships may exist. For example, A and / or B may represent three cases: only A exists, both A and B exist, and only B exists.

[0080] In the description and claims of the embodiments of the present application, terms such as "first," "second," etc. are intended to distinguish between different objects, and do not indicate a particular order of the objects. For example, a first target object, a second target object, etc. are used to distinguish between different target objects, and are not used to describe a particular order of the target objects.

[0081] In the embodiments of the present application, words such as "example" and "for example" are used to denote an example, instance, or description. Any embodiment or design solution described in the embodiments of the present application as "example" or "for example" should not be construed as preferred or advantageous over other embodiments or design solutions. Indeed, words such as "example" and "for example" are used to concretely present related concepts.

[0082] In describing the embodiments of the present application, "plurality" means two or more unless otherwise specified. For example, a plurality of processing units refers to two or more processing units, and a plurality of systems refers to two or more systems.

[0083] For example, embodiments of the present application may be applied to an N-end (ie, N devices) synergistic rendering scene, where N is an integer greater than 1.

[0084] In a possible scene, one device may generate rendering input information (the rendering input information may include one or more of a three-dimensional model (which may be referred to as a 3D (3-dimensional) model) including a human model and / or an object model, probe data, etc., which is not limited in this application, and an example in which the rendering input information is probe data is used for explanation in this application). The probe data is distributed to the other N-1 devices. After receiving the probe data, the other N-1 devices may determine shading effects of objects (corresponding to the three-dimensional model) in the three-dimensional scene based on the probe data in a rendering process. After rendering is completed, a rendered picture may be obtained.

[0085] In a possible scene, N1 devices (N1 ranges in value from 2 to N, and N1 may be equal to 2 or N, where N1 is an integer) may synergistically generate probe data. Each of the N1 devices generates a portion of the probe data. Then, each of the N1 devices distributes the portion of the probe data generated by that device to the other N-1 devices. After receiving the probe data, the N1 devices may determine, in a rendering process, shading effects of objects in the three-dimensional scene based on the received probe data and the portions of the probe data generated by the N1 devices. After rendering is completed, a rendered picture may be obtained. After receiving the probe data, the other N-N1 devices may determine, in a rendering process, shading effects of objects in the three-dimensional scene based on the received probe data. After rendering is completed, a rendered picture may be obtained.

[0086] For ease of description, a device that generates probe data in an N-end synergy rendering scene may be referred to as a first device, and a device that is used for rendering and determines shading effects of objects in a three-dimensional scene based on the probe data in the rendering process may be referred to as a second device. The same device may be the first device or the second device. This is not limited herein. The first device may be a server or a terminal. The second device may be a terminal.

[0087] 1a is a diagram of an example of a system framework. In the embodiment of FIG. 1a, the first device is a computing center server located on a cloud, and the second device is a client. FIG. 1a is a diagram of an example of a framework of a device-cloud synergy rendering system.

[0088] See FIG. 1a. For example, a device-cloud synergy rendering system may include a computing center server, an edge server, and a client. There may be n edge servers (n is an integer greater than 1) and k1 + k2 + ⋅ ⋅ + kn clients, where k1, k2, ⋅ ⋅ , and kn are all positive integers. The computing center server is connected to n edge servers, and each edge server is connected to at least one client. As shown in FIG. 1a, edge server 1 is connected to k1 clients, i.e., client 11, client 12, ⋅ ⋅ , and client k1; edge server 2 is connected to k2 clients, i.e., client 21, client 22, ⋅ ⋅ , and client k2; and edge server n is connected to kn clients, i.e., client n1, client n2, ⋅ ⋅ , and client kn.

[0089] For example, a client may access the network through a wireless access point, such as a base station or a Wi-Fi access point, and communicate with an edge server over the network. Alternatively, a client may communicate with an edge server through a wired connection. An edge server may also access the network through a wireless access point, such as a base station or a Wi-Fi access point, and communicate with a computing center server over the network. Alternatively, an edge server may communicate with a computing center server over a wired connection.

[0090] For example, the computing center server may be a single server, a server cluster including multiple servers, or other distributed system, although this application is not limited thereto.

[0091] For example, the number n of edge servers is not limited in this application and may be specifically set based on an actual application scenario, which is not limited in this application. In a possible scenario, some clients may not be connected to edge servers and may be directly connected to computing center servers. In another possible scenario, all clients may not be connected to edge servers and may be directly connected to computing center servers.

[0092] For example, the number of clients connected to each edge server is not limited in the present application and may be specifically set based on the actual application scenario. Furthermore, the numbers of clients connected to all edge servers may be the same or different (in other words, k1, k2, ... and kn may be equal or unequal), and may be specifically set based on the actual application scenario. This is not limited in the present application.

[0093] For example, the client may include, but is not limited to, a mobile phone, a personal computer (PC), a notebook computer, a tablet computer, a smart television, a mobile Internet device (MID), a wearable device (such as a smart watch, smart glasses, or smart helmet), a virtual reality (VR) device, an augmented reality (AR) device, a wireless electronic device in industrial control, a wireless electronic device in self-driving, a wireless electronic device in remote medical surgery, a wireless electronic device in a smart grid, a wireless electronic device in transportation safety, a wireless electronic device in a smart city, a wireless electronic device in a smart home, etc. The specific form of the client is not particularly limited in the following embodiments.

[0094] It should be understood that the framework of the device-cloud synergy rendering system shown in FIG. 1a is just one example of the framework of the device-cloud synergy rendering system of the present application. In the device-cloud synergy rendering system of the present application, the computing center server and the edge server may be the same server. Alternatively, the device-cloud synergy rendering system of the present application does not include an edge server, and the computing center server is connected to each client. This is not a limitation of the present application. In the present application, the framework of the device-cloud synergy rendering system shown in FIG. 1a is used as an example for description.

[0095] For example, the computing center server may be configured to generate probe data, for example, probe data shared by all clients and probe data personalized by each client.

[0096] For example, an edge server may be configured to distribute probe data and generate personalized probe data for clients connected to the edge server. For example, when a computing center server generates probe data shared by all clients and personalized probe data for each client, the edge server may distribute the probe data generated by the computing center server. As another example, when a computing center server generates only probe data shared by all clients, the edge server may generate personalized probe data for clients connected to the edge server, and then distribute the probe data generated by the computing center server and the probe data generated by the edge server.

[0097] For example, the client may be used for rendering and configured to display the rendered picture. During the rendering process, shading effects for objects in the three-dimensional scene may be determined based on the probe data.

[0098] For example, all N-end synergy rendering scenes, such as cloud gaming, cloud exhibition, interior decoration, clothing design, and architectural design, can be implemented by using the framework of the device-cloud synergy rendering system shown in Figure 1a.

[0099] For example, in a cloud game scene, after receiving a viewing angle switching command sent by the client 11, the computing center server may generate probe data of the game scene corresponding to the target viewing angle, and then send the probe data to the edge server 1. The edge server 1 sends the probe data to the client 11. After receiving the probe data, the client 11 may perform rendering, and in the rendering process, determine the shading effect of the object in the game scene corresponding to the target viewing angle based on the received probe data, and after completing the rendering, obtain and display a picture of the game scene corresponding to the target viewing angle.

[0100] For example, in an indoor decoration scene, after receiving a furniture addition command sent by the client 21, the computing center server may generate probe data corresponding to the living room scene to which the target furniture is to be added, and then send the probe data to the edge server 2. The edge server 2 sends the probe data to the client 21. After receiving the probe data, the client 21 may perform rendering, and in the rendering process, determine the shading effect of the objects in the living room scene to which the target furniture is to be added based on the received probe data, and obtain and display a picture of the living room to which the target furniture is to be added.

[0101] For ease of explanation, the following describes the process in which the computing center server generates probe data, and the process in which the client determines the shading effect of an object in a three-dimensional scene based on the probe data in the rendering process.

[0102] The process by which the computing center server generates probe data is as follows.

[0103] For example, the rendering process of the computing center server may be as follows: load a three-dimensional model (which may include a human model or an object model) into a three-dimensional scene (which may also be referred to as a 3D scene) (thus converting the three-dimensional model into an object in the three-dimensional scene), and render the object in the three-dimensional scene to obtain a current frame (i.e., a rendered picture). In order to simulate the shading effect of the object in the three-dimensional scene after a light ray is reflected multiple times in the three-dimensional scene, multiple probes may be placed in the three-dimensional scene during the process of rendering the object in the three-dimensional scene, and the surrounding environment is detected by the probes to obtain probe data. Then, the shading effect of the object in the three-dimensional scene is determined based on the probe data.

[0104] Figure 1b is a diagram of an example of probe distribution in a 3D scene. Each small ball in Figure 1b represents a probe. In the embodiment of Figure 1b, the probes are dynamic diffuse global illumination (DDGI) probes.

[0105] See Figure 1b. For example, the position where each probe is placed in the three-dimensional scene and the positional relationship between each probe and other probes can be set based on requirements, and this is not limited by the present application. For example, in Figure 1b, the distances between each probe and its six surrounding probes in six directions (upper right, lower right, front right, rear right, left right, and right right) are equal. Furthermore, the number of probes placed in the three-dimensional scene can also be set based on requirements, and this is not limited by the present application.

[0106] After multiple probes are placed in a three-dimensional scene, corresponding attribute data (which is used in the rendering process) is set for each probe based on scene requirements. The attribute data includes, but is not limited to, the type of probe (such as a reflection probe or a DDGI probe), the valid identifier of the probe, the position of the probe, and the position offset of the probe (for example, after the probes are placed in a preset manner, the initial position of the probe may be obtained, and the positions of some probes may be adjusted to obtain a better shading effect, whereby for these probes, the offset between the adjusted position and the initial position is called the position offset of the probe; for example, after the probes are placed as shown in FIG. 1b, the distances between each probe and its six surrounding probes are all equal, and when the position of the probe is adjusted, the distances between the probe and its six surrounding probes are no longer equal), etc. This is not limited in the present application.

[0107] For example, after multiple probes are placed in a three-dimensional scene, each probe may detect the surrounding environment around the probe, i.e., detect features of surrounding objects around the probe in the three-dimensional scene, and record these features as environmental data of the probe. The environmental data may include illuminance data, color data, visibility data, normal direction, texture coordinates, material information, etc., and the illuminance data may be used to describe the radiative illumination of the objects surrounding the probe. The visibility data may be used to describe the distribution of distances between the objects surrounding the probe and the probe (also referred to as a depth distribution), and may include, but is not limited to, distance data, the variance of the distance data, the square of the distance data, etc. This is not a limitation of the present application. The following uses an example in which illuminance data and visibility data are generated for explanation.

[0108] For example, illumination data and visibility data corresponding to each probe may be generated according to the DDGI algorithm. The following uses the probe in the current frame as an example to describe the process of generating illumination data and visibility data for the probe. First, several rays emitted from the probe are sampled, and first intersection points between the several rays and an object in the three-dimensional scene are calculated. Then, distances between the probe and the first intersection points of the several rays of the probe and the object in the three-dimensional scene are calculated to obtain initial distance data. Illumination at the first intersection points of the several rays and the object in the three-dimensional scene is calculated to obtain initial illumination data. Thereafter, the initial distance data in the discrete domain may be converted into spherical data in the continuous domain. Specifically, a filtering process may be performed on the initial distance data by using a cos^k kernel function (k is a positive integer) on the sphere to obtain candidate distance data. Furthermore, the initial distance data in the discrete domain may be converted into spherical data in the continuous domain. Specifically, a filtering process may be performed on the square of the initial distance data by using a cos^k kernel function (k is a positive integer) on a spherical surface to obtain the square of the candidate distance data. Furthermore, the initial illumination data in the discrete domain may be converted into spherical data in a continuous domain. Specifically, a filtering process may be performed on the initial illumination data by using a cos kernel function on a spherical surface to obtain the candidate illumination data. Then, a weighting calculation may be performed on the candidate distance data of the probe and the distance data of the probe in the previous frame to obtain the distance data of the probe in the current frame. A weighting calculation may be performed on the square of the candidate distance data of the probe and the square of the distance data of the probe in the previous frame to obtain the square of the distance data of the probe in the current frame. A weighting calculation may be performed on the candidate illumination data of the probe and the illumination data of the probe in the previous frame to obtain the illumination data of the probe in the current frame. In this way, the illumination data and visibility data of all the probes in the current frame may be obtained.

[0109] For example, the illumination data and visibility data of each probe can be represented by a two-dimensional picture, or spherical harmonic basis coefficients, or spherical wavelet basis coefficients, which is not limited in this application.

[0110] It should be noted that a 3D scene is assumed to include M probes (M is a positive integer). M1 probes have any one of illuminance data, visibility data, and attribute data, M2 probes have any two of illuminance data, visibility data, and attribute data, M3 probes have illuminance data, visibility data, and attribute data, and M4 probes have no probe data. M1+M2+M3+M4=M, where M1, M2, M3, and M 4 are all integers, and the values ​​of M1, M2, M3 and M4 can be set based on requirements, which is not limited in this application.

[0111] For example, the probe data may include attribute data and environmental data that are used in the rendering process.

[0112] The process by which the client determines the shading effect of an object in a 3D scene based on the probe data during the rendering process is as follows.

[0113] For example, when a client is in the rendering process, the probe data can be used to calculate the shading effect of an object in a three-dimensional scene. Specifically, when each pixel is rendered, the coordinate in 3D space corresponding to the pixel is first obtained, and then eight probes around the coordinate are searched for. Next, the weight contributed by each probe to the pixel is calculated based on the visibility data of the probe, that is, whether the probe and the 3D coordinate of the probe are mutually visible is determined based on the distance. If the probe and the 3D coordinate of the probe are mutually visible, the weight is 0. If the probe and the 3D coordinate of the probe are not mutually visible, the weight contributed to the probe is calculated based on the distance, the square of the distance, and the position of the probe. The contributed weight is then used to perform weighted averaging on the illumination data of the probe to obtain the shading result of the pixel.

[0114] Because the amount of probe data is large, the computing center server can compress the probe data before sending it to the client in order to reduce server and client bandwidth usage and client rendering delays.

[0115] FIG. 2a is a diagram of an example coding framework.

[0116] See Figure 2a. For example, the encoder may include a bitstream load balancing module, a data format conversion module, a first rearrangement module, and a coding module.

[0117] For example, the bitstream load balancing module may be configured to determine a target bit rate and encoding scheme (eg, intra-frame encoding or inter-frame encoding) for the probe data.

[0118] For example, the data format conversion module may be configured to perform data format conversion on the environment data in order to convert it into a more compact representation format or to increase the number of bits occupied in the bitstream by more important data required in the rendering process.

[0119] For example, the first rearrangement module may be configured to rearrange attribute data of the probe, which may include attribute data used for data format conversion (referred to as first attribute data) and attribute data used in the rendering process (referred to as second attribute data).

[0120] For example, the encoding module is configured to perform encoding to obtain a bitstream.

[0121] It should be noted that the steps performed by the bitstream load balancing module, the data format conversion module, and the first rearrangement module are steps in the encoding procedure of the encoder.

[0122] 2a is merely an example of an encoder of the present application, and the encoder of the present application may have fewer modules than those shown in Fig. 2a. For example, the encoder may include a bitstream load balancing module, a data format conversion module, and an encoding module. As another example, the encoder may include a data format conversion module, a first rearrangement module, and an encoding module. As yet another example, the encoder may include a data format conversion module and an encoding module. Furthermore, the encoder of the present application may have more modules than those shown in Fig. 2a. This is not a limitation of the present application.

[0123] It should be understood that the bitstream load balancing module, data format conversion module, first rearrangement module, and encoding module in FIG. 2a may be independent modules, or any two or more modules may be integrated into a whole. This is not a limitation of the present application. Furthermore, the bitstream load balancing module, data format conversion module, first rearrangement module, and encoding module are logic modules. Alternatively, the encoder may be divided into other modules. Alternatively, these modules may use other names. This is also not a limitation of the present application.

[0124] It should be understood that in a possible embodiment, the encoder only includes a coding module, a bitstream load balancing module, a data format conversion module, and a first rearrangement module, which may be independent of the encoder, and this is not limited in this application. In this application, the encoder of Fig. 2a is used as an example for explanation.

[0125] FIG. 2b is a diagram of an example of the structure of a data format conversion module.

[0126] See Figure 2b. For example, the data format conversion module may include a quantization module, a domain conversion module, and a second rearrangement module.

[0127] For example, the quantization module may be configured to quantize the probe data.

[0128] For example, the domain transformation module may be configured to perform a domain transformation on the probe data.

[0129] For example, domain transformation may refer to converting the representation of data from one domain to another. Domains can be classified from different perspectives based on requirements. Examples are as follows:

[0130] From a normalization perspective, domains can be classified into normalized and non-normalized regions.

[0131] From a color point of view, the domain can be divided into the RGB domain, the YUV domain, the XYZ domain, and the Lab domain.

[0132] From the point of view of numerical relations, the domains can be classified into linear and nonlinear domains. The nonlinear domains can be exponential domain, PQ (perceptual quantizer) domain, HLG (hybrid log gamma) domain, etc.

[0133] From the perspective of numerical representation format, the domain can be classified into a picture domain and a transform domain. For example, the picture domain may be a domain represented by a picture. For example, the transform domain may be a domain represented by transform basis functions and corresponding transform coefficients. For data Y(t) in the transform domain, an approximation can be performed on the data Y(t) by using x transform bases e_1(t) to e_x(t), so that the data Y(t) is approximately equal to the sum of the products of the x transform bases and the corresponding transform coefficients. The transform bases include, but are not limited to, spherical harmonic bases, spherical wavelet bases, eigenvectors, etc. This is not limited in this application.

[0134] For example, the second relocation module may be configured to relocate the data.

[0135] It should be understood that Figure 2b is only one example of a data format conversion module of the present application. The data format conversion module of the present application may have fewer modules than those shown in Figure 2b. For example, the data format conversion module may include only a domain conversion module. As another example, the data format conversion module may include only a quantization module and a domain conversion module. As yet another example, the data format conversion module may include only a domain conversion module and a second rearrangement module. This is not a limitation of the present application. Furthermore, the data format conversion module of the present application may include more modules than those shown in Figure 2b. This is also not a limitation of the present application.

[0136] It should be understood that the quantization module, domain transformation module, and second rearrangement module in FIG. 2b may be independent modules, or any two or more modules may be integrated into a whole. This is not a limitation of the present application. Furthermore, the quantization module, domain transformation module, and second rearrangement module are logic modules. The data format conversion module may alternatively be divided into other modules. Alternatively, these modules may use other names. This is also not a limitation of the present application.

[0137] The following describes an example of the encoding process of the first device.

[0138] 3a is a diagram of an example of an encoding procedure. The encoding procedure may include: S301: obtaining data; S302: performing data format conversion; and S303: performing encoding.

[0139] S301: Data is acquired, that is, probe data is acquired.

[0140] For example, in an N-end synergy rendering scenario, the first device may generate probe data for K (K is a positive integer, and K is less than or equal to M) probes in the current frame based on the above description, and then input the probe data of the K probes to the encoder. In this way, the encoder can obtain the probe data of the K probes, and then encode the probe data of the K probes with reference to the following S302 and S303 to obtain a bitstream corresponding to the probe data.

[0141] For example, in the embodiment of Figure 3a, the probe data for any probe may include illuminance data and / or visibility data.

[0142] S302: Perform data format conversion, that is, perform data format conversion on the probe data acquired in S301 to acquire intermediate data.

[0143] For example, after probe data of K probes is received, data format conversion may be performed on the probe data of K probes, that is, the probe data of K probes is converted from one data format to another data format. In this way, the probe data of K probes can be converted into a more compact representation format, which facilitates compression. Furthermore, compared with the prior art, the bit rate can be reduced under the same rendering effect. Also, the number of bits occupied in the bitstream by data with higher importance required in the rendering process can be increased. Thus, compared with the prior art, the rendering effect is improved under the same bit rate.

[0144] For example, data format conversion may include domain conversion.

[0145] For example, the domain transformation may include at least one of the following: a transformation from a non-normalized domain to a normalized domain, a transformation from a linear domain to a non-linear domain, a transformation from an RGB domain to a YUV domain, a transformation from an RGB domain to an XYZ domain, a transformation from an RGB domain to an Lab domain, and a transformation from a picture domain to a transform domain. It should be understood that the present application may also include other types of domain transformations, which are not limited herein.

[0146] For example, a transformation from a non-normalized domain to a normalized domain is a normalization. When the domain transformation is a transformation from a non-normalized domain to a normalized domain, the probe data of the K probes may be normalized based on a normalization parameter to obtain intermediate data corresponding to the probe data of the K probes. For example, the probe data of the K probes may be normalized to [0,1]. In this way, the intermediate data corresponding to the probe data of the K probes belong to [0,1].

[0147] Normalization may include, but is not limited to, adaptive normalization, fixed parameter normalization, min-max normalization, z-score normalization, etc. This is not a limitation of the present application. It should be understood that the present application may also include other types of normalization. This is not a limitation of the present application.

[0148] In this way, the probe data is converted into a fixed range by normalizing the probe data, so that the values ​​of the probe data are more centralized, that is, the probe data is converted into a more compact representation format, which makes compression easier. In this way, compared with the prior art, the bit rate can be reduced under the same rendering effect.

[0149] For example, the acquired probe data of the K probes is data in a linear domain. If the nonlinear domain is an exponential domain, a transformation from the linear domain to the exponential domain can be performed on the probe data of the K probes based on the exponential transformation parameters to obtain intermediate data corresponding to the probe data of the K probes. In this case, the intermediate data corresponding to the probe data of the K probes belongs to the exponential domain.

[0150] For example, if the nonlinear domain is the PQ domain, a transformation from the linear domain to the PQ domain may be performed on the probe data of the K probes by using the PQ curve to obtain intermediate data corresponding to the probe data of the K probes, where the intermediate data corresponding to the probe data of the K probes belongs to the PQ domain.

[0151] For example, if the nonlinear domain is the HLG domain, conversion from the linear domain to the HLG domain may be performed on the probe data of the K probes by using the HLG curve to obtain intermediate data corresponding to the probe data of the K probes, where the intermediate data corresponding to the probe data of the K probes belongs to the HLG domain.

[0152] In this case, the conversion from linear domain to nonlinear domain is performed on the probe data so that the number of bits occupied in the bitstream by the more important data required in the rendering process can be increased, thus, compared with the prior art, the rendering effect is better under the same bit rate.

[0153] For example, the acquired probe data of the K probes is data expressed in RGB. In a possible embodiment, a conversion from the RGB domain to the YUV domain may be performed on the probe data of the K probes by converting the RGB color space to the YUV 444 color space to obtain intermediate data corresponding to the probe data of the K probes. In this case, the intermediate data corresponding to the probe data of the K probes is expressed in YUV.

[0154] In a possible embodiment, a conversion from the RGB domain to the YUV domain may be performed on the probe data of the K probes by converting from the RGB color space to the YUV 420 color space to obtain intermediate data corresponding to the probe data of the K probes, where the intermediate data corresponding to the probe data of the K probes is represented in YUV.

[0155] For example, a conversion from the RGB domain to the XYZ domain may be performed on the probe data of the K probes by converting the RGB color space to the XYZ color space to obtain intermediate data corresponding to the probe data of the K probes, i.e., the intermediate data corresponding to the probe data of the K probes is expressed in terms of XYZ.

[0156] For example, a conversion from the RGB domain to the Lab color space may be performed on the probe data of the K probes by converting the RGB color space to the Lab color space to obtain intermediate data corresponding to the probe data of the K probes, i.e., the intermediate data corresponding to the probe data of the K probes is expressed in Lab.

[0157] Thus, the redundancy of the probe data of the probes can be removed by performing a conversion from the RGB color space to the YUV region on the probe data of K probes. Further, compared with the prior art, the bit rate can be reduced under the same rendering effect. Alternatively, a conversion from the RGB region to the XYZ region / Lab region is performed on the probe data of K probes, whereby the data can better satisfy the perception of the human eye. Further, compared with the prior art, the rendering effect can be improved under the same bit rate.

[0158] For example, when the acquired probe data of K probes is represented by a two-dimensional picture, the probe data of K probes may be decomposed (for example, decomposed by the PCA (principal component analysis) method). In this case, in order to perform a conversion from the picture region to the conversion region on the probe data of K probes, the probe data of each probe is approximately equal to the sum of the products of x conversion bases e_1(t) to e_x(t) and the corresponding conversion coefficients.

[0159] For example, theoretically, the probe data of each probe is equal to the sum of the products of infinite or y conversion bases and the corresponding conversion coefficients. However, in the present application, only the sum of the products of a limited number of x (x < y) conversion bases e_1(1) to e_x(t) and the corresponding conversion coefficients is used to approximately represent the probe data of each probe. Essentially, the high-order bases and the corresponding coefficients are discarded. The influence of the high-order bases and coefficients on the rendering effect is slight. In this case, the conversion from the picture region to the conversion region is performed on the probe data so as to reduce the amount of data that needs to be compressed. Further, compared with the prior art, the bit rate can be reduced under the same rendering effect.

[0160] It should be understood that two or more types of domain transformations among the above multiple types of domain transformations can be used to implement the data format transformation of the probe data. When two or more types of domain transformations are used to implement the data format transformation of the probe data, the order and number of times that the various types of domain transformations are performed are not limited by this application.

[0161] For example, it is assumed that normalization and conversion from picture domain to transform domain are used to implement the data format conversion of the probe data. In a possible manner, the probe data may be normalized and a conversion from picture domain to transform domain performed on the normalized probe data to obtain intermediate data. In a possible manner, a conversion from picture domain to transform domain may be performed on the probe data and then the probe data in the transform domain is normalized to obtain intermediate data.

[0162] As another example, it is assumed that normalization, conversion from the RGB domain to the YUV domain, and conversion from the linear domain to the exponential domain are used to implement data format conversion of the probe data. In a possible embodiment, the probe data may be normalized, a conversion from the linear domain to the exponential domain is performed on the normalized probe data, and then a conversion from the RGB domain to the YUV domain is performed on the data in the exponential domain to obtain intermediate data. In a possible embodiment, the probe data may be normalized, a conversion from the RGB domain to the YUV domain is performed on the normalized probe data, and a conversion from the linear domain to the exponential domain is performed on the data in the YUV domain to obtain intermediate data. In a possible embodiment, normalization may be performed on the probe data first based on a normalization parameter, a second normalization may be performed on the normalized probe data based on another normalization parameter, the probe data obtained by the two normalizations is converted from the RGB domain to the YUV domain, and a conversion from the linear domain to the exponential domain is performed on the data in the YUV domain to obtain intermediate data.

[0163] S303: Perform encoding, ie encode the intermediate data obtained in S302 to obtain a corresponding bitstream.

[0164] For example, the 3D scenes of adjacent frames may be the same, so the probe data of adjacent frames may be related to a certain extent. Therefore, if the 3D scene of the current frame is the same as the 3D scene of the previous frame, inter-frame coding may be performed on the intermediate data corresponding to the probe data of the K probes in the current frame to obtain a corresponding bitstream. In this way, the bitrate may be reduced while the rendering effect is ensured. If the 3D scene of the current frame is different from the 3D scene of the previous frame, intra-frame coding may be performed on the intermediate data corresponding to the probe data of the K probes in the current frame to ensure the rendering effect.

[0165] For example, any video encoding scheme may be used to encode the intermediate data to obtain a bitstream corresponding to the probe data. For example, HEVC (high efficiency video coding) encoding may be performed on the intermediate data to obtain the corresponding bitstream. As another example, AVC (advanced video coding) encoding may be performed on the intermediate data to obtain the corresponding bitstream. As yet another example, VVC (versatile video coding) encoding may be performed on the intermediate data to obtain the corresponding bitstream. As yet another example, entropy encoding may be performed on the intermediate data to obtain the corresponding bitstream. As yet another example, the intermediate data may alternatively be encoded with another video encoding scheme, such as AVS (audio video coding standard), to obtain the corresponding bitstream.

[0166] It should be understood that the video encoding scheme for the intermediate data is not limited in this application, and lossy or lossless compression may be performed on the intermediate data, which can be specifically set based on requirements, which is also not limited in this application.

[0167] For example, if the probe data includes illuminance data, the bit stream acquired in S303 is a bit stream corresponding to the illuminance data. If the probe data includes visibility data, the bit stream acquired in S303 is a bit stream corresponding to the visibility data. If the probe data includes illuminance data and visibility data, the bit stream acquired in S303 includes a bit stream corresponding to the illuminance data and a bit stream corresponding to the visibility data.

[0168] It should be noted that the step performed in S302 is a step within the encoding procedure.

[0169] In this way, the probe data is converted into a more compact representation format by performing a data format conversion on the probe data. Compared with the prior art, the bit rate can be reduced under the same rendering effect. Alternatively, the data format conversion is performed on the probe data so as to increase the number of bits in the bitstream occupied by data that is more important in the rendering process. Compared with the prior art, the rendering effect is improved under the same bit rate.

[0170] FIG. 3b is a diagram of an example encoding procedure.

[0171] See Figure 3b(1). For example, the acquired probe data is illuminance data. A conversion from the RGB domain to the YUV domain is performed on the illuminance data to obtain intermediate data. Then, HEVC encoding (or other encoding) is performed on the intermediate data to obtain a bitstream corresponding to the illuminance data.

[0172] See Figure 3b(2). For example, the acquired probe data is illuminance data. A linear to exponential domain conversion is performed on the illuminance data to obtain intermediate data. Then, HEVC encoding (or other encoding) is performed on the intermediate data to obtain a bitstream corresponding to the illuminance data.

[0173] See Figure 3b(3). For example, the acquired probe data is illuminance data. A transformation from the picture domain to a transform domain is performed on the illuminance data to obtain intermediate data. Then, entropy coding (or other coding) is performed on the intermediate data to obtain a bitstream corresponding to the illuminance data.

[0174] See Figure 3b(4). For example, the acquired probe data is visibility data. The visibility data is normalized to obtain intermediate data. Then, entropy coding (or other coding) is performed on the intermediate data to obtain a bitstream corresponding to the visibility data.

[0175] See Figure 3b(5). For example, the acquired probe data is illuminance data. The illuminance data is normalized, and a conversion from the RGB domain to the YUV domain is performed on the illuminance data to obtain intermediate data. Then, entropy coding (or other coding) is performed on the intermediate data to obtain a bitstream corresponding to the illuminance data.

[0176] See Figure 3b(6). For example, the acquired probe data includes illumination data and visibility data. The illumination data and visibility data are normalized, and a transformation from the picture domain to a transform domain is performed on the illumination data and visibility data to obtain intermediate data. Then, entropy coding (or other coding) is performed on the intermediate data to obtain a bitstream corresponding to the illumination data and a bitstream corresponding to the visibility data.

[0177] For example, the data format conversion may further include quantization and / or rearrangement. Thus, data format conversion may be performed on the probe data by combining a domain transformation and quantization, by combining a domain transformation and rearrangement, or by combining a domain transformation and quantization and rearrangement. Both quantization and rearrangement can further convert the data into a more compact representation format, and then data format conversion is performed on the probe data by combining a domain transformation, quantization, and rearrangement, thereby further reducing the bit rate.

[0178] When a combination of domain transformation, quantization, and / or rearrangement is used, the order in which the domain transformation, quantization, and rearrangement are performed is not limited herein, nor is the number of times any one of the domain transformation, quantization, and rearrangement processing modes is performed.

[0179] For example, quantization may include uniform quantization or non-uniform quantization. Non-uniform quantization may include, but is not limited to, μ-law quantization, a-law quantization, etc., without limitation herein. Alternatively, quantization may further include adaptive quantization and other types of quantization, without limitation herein.

[0180] For example, rearrangement can include first-mode rearrangement and / or second-mode rearrangement.

[0181] For example, if the probe data is illuminance data, and if the illuminance data includes multiple channels, the first manner rearrangement includes at least one of the following: discarding data of some channels, converting the illuminance data to a preset precision, or performing a dimensional transformation on the illuminance data.

[0182] It should be noted that the "illuminance data" in "converting the illuminance data to a preset precision" may be the original illuminance data, the illuminance data obtained by domain transformation, the illuminance data obtained by quantization, the illuminance data obtained by discarding data of some channels, or the illuminance data obtained by dimension transformation.

[0183] It should be noted that the "illuminance data" in "performing a dimensional transformation on the illuminance data" may be the original illuminance data, the illuminance data obtained by a dimensional transformation, the illuminance data obtained by quantization, the illuminance data obtained by discarding data of some channels, or the illuminance data converted to a preset precision.

[0184] For example, if the probe data is visibility data and the visibility data includes multiple channels, the first modality rearrangement includes at least one of the following: performing channel splitting, converting the visibility data to a preset precision, or performing a dimensional transformation on the visibility data.

[0185] It should be noted that the "visibility data" in "converting visibility data to a preset precision" may be original visibility data, visibility data obtained by domain transformation, visibility data obtained by quantization, visibility data obtained by channel splitting, or visibility data obtained by dimension transformation.

[0186] It should be noted that the "visibility data" in "performing a dimensional transformation on the visibility data" may be the original visibility data, visibility data obtained by dimensional transformation, visibility data obtained by quantization, visibility data obtained by channel splitting, or visibility data converted to a preset precision.

[0187] For example, the second modality rearrangement may be used to pad data (which may be probe data obtained by domain transformation, quantized probe data, or probe data obtained by first modality rearrangement) into a YUV plane.

[0188] The following describes several ways to combine domain transformation, quantization, and / or rearrangement.

[0189] FIG. 4a is a diagram of an example encoding procedure.

[0190] In the embodiment of Fig. 4a(1), a method of combining domain transformation and quantization is described. See Fig. 4a(1). For example, after probe data is obtained, domain transformation may first be performed on the probe data to obtain transformed data 1, and then the transformed data 1 is quantized to obtain intermediate data. After that, the intermediate data is encoded to obtain a corresponding bitstream.

[0191] In the embodiment of Fig. 4a(2), a method of combining domain transformation and rearrangement is described. See Fig. 4a(2). For example, after probe data is obtained, domain transformation may first be performed on the probe data to obtain transformed data 1, and then the transformed data 1 is rearranged to obtain intermediate data. The intermediate data is then encoded to obtain a corresponding bitstream. The rearrangement in Fig. 4a(2) may include first-style rearrangement and / or second-style rearrangement.

[0192] In the embodiment of Fig. 4a(3), a method of combining domain transformation, quantization, and rearrangement is described. See Fig. 4a(3). For example, after probe data is obtained, domain transformation may first be performed on the probe data to obtain transformed data 1, the transformed data 1 is quantized to obtain transformed data 2, and then the transformed data 2 is rearranged to obtain intermediate data. After that, the intermediate data is encoded to obtain a corresponding bitstream.

[0193] In the embodiment of Fig. 4a(4), a method of combining domain transformation, quantization, and first modality rearrangement is described. See Fig. 4a(4). For example, after probe data is obtained, domain transformation may first be performed on the probe data to obtain transformed data 1, and first modality rearrangement is performed on the transformed data 1 to obtain transformed data 3, and then the transformed data 3 is quantized to obtain intermediate data. After that, the intermediate data is encoded to obtain a corresponding bitstream.

[0194] In the embodiment of Fig. 4a(5), a method of combining domain transformation, quantization, and rearrangement is described. See Fig. 4a(5). For example, after probe data is obtained, domain transformation may first be performed on the probe data to obtain transformed data 1, a first modality rearrangement is performed on the transformed data 1 to obtain transformed data 3, the transformed data 2 is quantized to obtain transformed data 4, and then a second modality rearrangement is performed on the transformed data 4 to obtain intermediate data. The intermediate data is then encoded to obtain a corresponding bitstream.

[0195] FIG. 4b is a diagram of an example encoding procedure.

[0196] In the embodiment of Fig. 4b(1), a method of combining domain transformation and quantization is described. See Fig. 4b(1). For example, after probe data is obtained, the probe data may first be quantized to obtain transformed data 5, and then domain transformation is performed on the transformed data 5 to obtain intermediate data. After that, the intermediate data is encoded to obtain a corresponding bitstream.

[0197] In the embodiment of Fig. 4b(2), a method of combining domain transformation, quantization, and rearrangement is described. See Fig. 4b(2). For example, after probe data is obtained, the probe data may first be quantized to obtain transformed data 5, a domain transformation is performed on the transformed data 5 to obtain transformed data 6, and then the transformed data 6 is rearranged to obtain intermediate data. The intermediate data is then encoded to obtain a corresponding bitstream. The rearrangement in Fig. 4b(2) may include a first modality rearrangement and / or a second modality rearrangement.

[0198] In the embodiment of Fig. 4b(3), a method of combining domain transformation, quantization, and rearrangement is described. See Fig. 4b(3). For example, after probe data is obtained, the probe data may first be quantized to obtain transformed data 5, a first modality rearrangement is performed on the transformed data 5 to obtain transformed data 7, and then a domain transformation is performed on the transformed data 7 to obtain intermediate data. The intermediate data is then encoded to obtain a corresponding bitstream.

[0199] In the embodiment of Fig. 4b(4), a method of combining domain transformation, quantization, and rearrangement is described. See Fig. 4b(4). For example, after probe data is obtained, the probe data may first be quantized to obtain transformed data 5, a first modality rearrangement is performed on the transformed data 5 to obtain transformed data 7, a domain transformation is performed on the transformed data 7 to obtain transformed data 8, and then a second modality rearrangement is performed on the transformed data 8 to obtain intermediate data. The intermediate data is then encoded to obtain a corresponding bitstream.

[0200] Figures 4c(1), 4c(2), 4c(3), 4c(4), 4c(5), 4c(6), and 4c(7) are diagrams of examples of encoding procedures.

[0201] In the embodiment of Fig. 4c(1), a method of combining domain transformation and rearrangement is described. See Fig. 4c(1). For example, after probe data is obtained, a first modality rearrangement may first be performed on the probe data to obtain transformed data 9, and then a domain transformation is performed on the transformed data 9 to obtain intermediate data. The intermediate data is then encoded to obtain a corresponding bitstream.

[0202] In the embodiment of Fig. 4c(2), a method of combining domain transformation, rearrangement, and quantization is described. See Fig. 4c(2). For example, after probe data is obtained, a first modality rearrangement may first be performed on the probe data to obtain transformed data 9, a domain transformation is performed on the transformed data 9 to obtain transformed data 10, and then the transformed data 10 is quantized to obtain intermediate data. The intermediate data is then encoded to obtain a corresponding bitstream.

[0203] In the embodiment of Fig. 4c(3), a method of combining domain transformation, rearrangement, and quantization is described. See Fig. 4c(3). For example, after probe data is obtained, a first modality rearrangement may first be performed on the probe data to obtain transformed data 9, the transformed data 9 is quantized to obtain transformed data 11, and then a domain transformation is performed on the transformed data 11 to obtain intermediate data. The intermediate data is then encoded to obtain a corresponding bitstream.

[0204] In the embodiment of Fig. 4c(4), a method of combining domain transformation and rearrangement is described. See Fig. 4c(4). For example, after probe data is obtained, a first modality rearrangement may first be performed on the probe data to obtain transformed data 9, a domain transformation is performed on the transformed data 9 to obtain transformed data 10, and then a second modality rearrangement is performed on the transformed data 10 to obtain intermediate data. The intermediate data is then encoded to obtain a corresponding bitstream.

[0205] In the embodiment of Fig. 4c(5), a method of combining domain transformation, rearrangement, and quantization is described. See Fig. 4c(5). For example, after probe data is obtained, a first modality rearrangement may first be performed on the probe data to obtain transformed data 9, a domain transformation is performed on the transformed data 9 to obtain transformed data 10, the transformed data 10 is quantized to obtain transformed data 12, and then a second modality rearrangement is performed on the transformed data 12 to obtain intermediate data. The intermediate data is then encoded to obtain a corresponding bitstream.

[0206] In the embodiment of Fig. 4c(6), a method of combining domain transformation, rearrangement, and quantization is described. See Fig. 4c(6). For example, after probe data is obtained, a first modality rearrangement may first be performed on the probe data to obtain transformed data 9, the transformed data 9 is quantized to obtain transformed data 11, a domain transformation is performed on the transformed data 11 to obtain transformed data 13, and then a second modality rearrangement is performed on the transformed data 13 to obtain intermediate data. The intermediate data is then encoded to obtain a corresponding bitstream.

[0207] In the embodiment of Fig. 4c(7), a method of combining domain transformation, rearrangement, and quantization is described. See Fig. 4c(7). For example, after probe data is obtained, a first modality rearrangement may first be performed on the probe data to obtain transformed data 9, a domain transformation is performed on the transformed data 9 to obtain transformed data 10, a first modality rearrangement is performed on the transformed data 10 to obtain transformed data 14, a domain transformation is performed on the transformed data 14 to obtain transformed data 15, and then a second modality rearrangement is performed on the transformed data 15 to obtain intermediate data. The intermediate data is then encoded to obtain a corresponding bitstream.

[0208] 4c(7), it should be noted that the first modality rearrangement in performing the first modality rearrangement on the probe data to obtain transformed data 9 may include at least one of the following: discarding data of some channels, performing channel splitting, and converting data to a preset precision. The first modality rearrangement in performing the first modality rearrangement on the transformed data 10 to obtain transformed data 14 may include a dimensional transformation. The domain transformation in performing the domain transformation on the transformed data 9 to obtain transformed data 10 may be a transformation from the picture domain to the spherical harmonics domain. The domain transformation in performing the domain transformation on the transformed data 14 to obtain transformed data 15 may be a transformation from the spherical harmonics domain to the eigenvector domain.

[0209] It should be noted that the domain transformations in Figures 4a(1) to 4a(5), 4b(1) to 4b(4), and 4c(1) to 4c(6) may include at least one of the following: normalization, linear to nonlinear domain transformation, RGB to YUV domain transformation, RGB to XYZ domain transformation, RGB to Lab domain transformation, and picture to transform domain transformation. Also, the number of times different types of domain transformations are performed in Figures 4a(1) to 4a(5), 4b(1) to 4b(4), and 4c(1) to 4c(6), and the order in which different types of domain transformations are performed are not limited by this application.

[0210] It should be noted that the quantization in Figures 4a(1) to 4a(5), 4b(1) to 4b(4), and 4c(1) to 4c(6) may be uniform and / or non-uniform quantization, and the number of times and order in which different types of quantization are performed in Figures 4a(1) to 4a(5), 4b(1) to 4b(4), and 4c(1) to 4c(6) are not limited by this application.

[0211] It should be noted that the first-mode rearrangement in Figures 4a(1) to 4a(5), 4b(1) to 4b(4), and 4c(1) to 4c(6) may include at least one of the following: discarding channels, performing channel splitting, converting data to a preset precision, and performing dimensional transformation. Also, the number of times that different types of first-mode rearrangements are performed in Figures 4a(1) to 4a(5), 4b(1) to 4b(4), and 4c(1) to 4c(6) and the order in which different types of first-mode rearrangements are performed are not limited by this application.

[0212] The following describes in detail the process of encoding illuminance data by using the encoding procedure shown in FIG. 4c(5) as an example.

[0213] 5 is a schematic diagram of an example of an encoding procedure. The encoding procedure may include steps of: S501: obtaining data; S502: discarding channels; S503: performing precision conversion; S504: performing adaptive normalization; S505: performing linear-to-exponential domain conversion; S506: performing RGB-to-YUV domain conversion; S507: performing uniform quantization; S508: padding data to YUV planes; and S509: performing HEVC encoding. In the embodiment of FIG. 5, the illumination data is represented by a two-dimensional picture. The transformed data 9 includes transformed data 9a and transformed data 9b. The transformed data 10 includes transformed data 10a, transformed data 10b, and transformed data 10c.

[0214] S501: Obtain data, that is, obtain illuminance data represented by a two-dimensional picture.

[0215] For example, the illumination data of the K probes may be a two-dimensional picture where a single picture includes three channels of R, G, and B, or a two-dimensional picture where a single picture includes four channels of R, G, B, and A. Each probe may correspond to multiple pixels in the two-dimensional picture.

[0216] For example, the RGB format of each pixel is not limited in the present application and may be, for example, R11G11B10F (the R channel and the G channel are represented by an 11-bit floating-point number, and the B channel is represented by a 10-bit floating-point number), RGB9E5 (the R channel, the B channel, and the G channel are all represented by a 9-bit integer and share an exponent bit represented by a 5-bit integer), etc.

[0217] For example, the RGBA of each pixel is not limited in the present application and may be, for example, RGB10A2 (the R channel, the B channel, and the G channel are all represented by 10-bit integers, and the A channel is represented by a 2-bit integer), RGBA16F (the R channel, the B channel, the G channel, and the A channel are all represented by 16-bit floating-point numbers), etc.

[0218] S502: Discard a channel: If the illuminance data acquired in S501 includes four channels of R, G, B and A, the A channel of the illuminance data is discarded to obtain the converted data 9a.

[0219] For example, if the illuminance data includes four channels, R, G, B, and A, the A channel of the illuminance data is discarded to obtain the transformed data 9a, which is also a two-dimensional picture in which a single pixel includes three channels, R, G, and B.

[0220] For example, if the illuminance data includes three channels of R, G, and B, S502 does not need to be executed, and S503 is executed directly after S501 is executed.

[0221] S503: Execute precision conversion, that is, convert the converted data 9a obtained in S502 into a preset precision to obtain converted data 9b.

[0222] For example, the preset precision may be set to be equal to or greater than a precision threshold based on requirements to ensure the precision of the illuminance data sent to the second device and ensure the rendering effect. The precision threshold may be set based on requirements. If the precision threshold is a 16-bit floating-point number, the preset precision may be set to a 16-bit floating-point number or a 32-bit floating-point number. This is not limited in the present application.

[0223] For example, all the data of each channel (R channel, G channel, and B channel) of each pixel in the converted data 9a may be converted to a preset precision to obtain the converted data 9b, which is also a two-dimensional picture in which a single pixel includes three channels of R, G, and B, and the data of each channel has a preset precision.

[0224] Furthermore, since the precision of illuminance data generated in different application scenarios may be different, the illuminance data in different application scenarios can be converted to the same precision by performing S503, thereby facilitating subsequent data format conversion.

[0225] S504: Perform adaptive normalization, that is, perform adaptive normalization on the transformed data 9b obtained in S503 to obtain transformed data 10a.

[0226] In a possible embodiment, the maximum value of the R channel data of all pixels in the transformed data 9b may be determined, and the maximum value of the R channel data may be used as a normalization parameter. Then, the R channel data of each pixel in the transformed data 9b is divided by the normalization parameter to normalize the R channel data of each pixel in the transformed data 9b so that the R channel data of each pixel in the transformed data 9b is within [0,1]. Correspondingly, the G channel data and B channel data of all pixels in the transformed data 9b may be individually normalized in a manner that normalizes the R channel data of all pictures in the transformed data 9b so that the G channel data and B channel data of each picture in the transformed data 9b is within [0,1]. In this way, the transformed data 10a may be obtained. The transformed data 10a is also a two-dimensional picture in which a single pixel includes three channels, R, G, and B, and the data of each channel has a preset precision and belongs to [0,1].

[0227] In a possible embodiment, the maximum value of the R, G, and B channel data of all pixels in the transformed data 9b may be determined, and the maximum value of the R, G, and B channel data may be used as a normalization parameter. For a pixel in the transformed data 9b, to normalize the R, G, and B channel data of that pixel in the transformed data 9b, the R channel data of the pixel may be divided by the normalization parameter, the G channel data of the pixel may be divided by the normalization parameter, and the B channel data of the pixel may be divided by the normalization parameter. In this way, the R, G channel data, and B channel data of each pixel in the transformed data 9b may all be normalized to be within [0, 1]. In this way, the transformed data 10a may also be obtained.

[0228] The adaptive normalization of S504 may be replaced by any one of various types of normalization, such as fixed parameter normalization, min-max normalization, and z-score normalization, which is not limited in this application.

[0229] Fixed parameter normalization: A preset fixed parameter may be set in advance, and the preset fixed parameter may be used as a normalization parameter. For a pixel in the transformed data 9b, to normalize the R, G, and B three-channel data of the pixel in the transformed data 9b, the R channel data of the pixel may be divided by the normalization parameter, the G channel data of the pixel may be divided by the normalization parameter, and the B channel data of the pixel may be divided by the normalization parameter. In this way, the R, G channel data, and B three-channel data of each pixel in the transformed data 9b can all be normalized to be within [0, 1]. Further, transformed data 10a can also be obtained.

[0230] Min-max normalization: In a possible embodiment, the maximum and minimum values ​​of the R channel data of all pictures in the transformed data 9b may be determined, and the difference between the maximum and minimum values ​​of the R channel data is calculated to obtain a difference Q1. For a pixel in the transformed data 9b, the difference between the R channel data of the pixel and the minimum value is calculated to obtain a difference F1, and the difference F1 is divided by the difference Q1 to normalize the R channel data of that pixel in the transformed data 9b. In this way, the R channel data of each pixel in the transformed data 9b may be normalized to be within [0,1]. Correspondingly, the G channel data and B channel data of all pixels in the transformed data 9b may be individually normalized in a manner that normalizes the R channel data of all pixels in the transformed data 9b, so as to normalize the G channel data and B channel data of each pixel in the transformed data 9b to be within [0,1]. In this way, the transformed data 10a may be obtained.

[0231] In a possible embodiment, the maximum and minimum values ​​of the data of the three channels R, G, and B of all pictures in the transformed data 9b may be determined, and the difference between the maximum and minimum values ​​of the data of the three channels R, G, and B is calculated to obtain a difference Q2. For a pixel in the transformed data 9b, the difference between the data of the R channel of the pixel and the minimum value is calculated to obtain a difference F2, and the difference F2 is divided by the difference Q2 to normalize the data of the R channel of the pixel in the transformed data 9b. The difference between the data of the G channel of the pixel and the minimum value is calculated to obtain a difference F3, and the difference F3 is Converted data 9b The difference between the B channel data of the pixel and the minimum value is calculated to obtain a difference F4, which is then divided by the difference Q2 to normalize the B channel data of the pixel in the transformed data 9b. In this way, the data of the three channels of R, G, and B of each pixel in the transformed data 9b can all be normalized to be within [0, 1]. Further, transformed data 10a can also be obtained.

[0232] Z-score normalization: For a pixel in the transformed data 9b, the mean and standard deviation of the R channel data of all pixels in the transformed data 9b may be calculated, and then the R channel data of that pixel is normalized based on the mean and standard deviation. For example, z-score normalization can be performed using the following formula:

number

[0233] X is the R channel data of the pixel in the transformed data 9b, and X *is the z-scored data of the R channel of the pixel, E(X) is the average value of the R channel data of all pixels in the transformed data 9b, and √D(X) is the standard deviation of the R channel data of all pixels in the transformed data 9b.

[0234] Based on the above method, z-score normalization may be performed on the G channel data and B channel data of a pixel. Furthermore, in the above method, z-score normalization may be performed on each of the three channels of R, G, and B of all pixels in the transformed data 9b. Details will not be described again here.

[0235] S505: Perform a transformation from the linear domain to the exponential domain, that is, transform the transformed data 10a obtained in S504 from the linear domain to the exponential domain to obtain transformed data 10b.

[0236] For example, an exponential transformation parameter (which may be an exponent value such as 2.2 or 5.0) may be set in advance, and then, to transform the transformed data 10a from the linear domain to the exponential domain, a value corresponding to the exponent of the transformed data 10a may be calculated where the exponent of the transformed data 10a is the exponential transformation parameter. In this way, the transformed data 10b is obtained. The transformed data 10b is also a two-dimensional picture in which a single pixel includes three channels, R, G, and B, the data of each channel has a preset precision, and the data of each channel is data in the exponential domain.

[0237] It should be understood that the conversion from the linear domain to the exponential domain in S505 may be replaced with the PQ domain or the HLG domain, which is not limited in this application.

[0238] For example, if S505 is to convert the transformed data 10a from the linear domain to the PQ domain to obtain the transformed data 10b, the transformed data 10a can be mapped to the PQ domain by using a PQ curve to obtain the transformed data 10b. In this case, the transformed data 10b is also a two-dimensional picture in which a single pixel includes three channels, R, G, and B, and the data of each channel has a preset precision, and the data of each channel is data in the PQ domain. For example, the data of each of the three channels, R, G, and B, of each picture in the two-dimensional picture can be mapped by using a PQ curve to obtain the transformed data 10b.

[0239] For example, if S505 is to convert the transformed data 10a from the linear domain to the HLG domain to obtain the transformed data 10b, the transformed data 10a can be mapped to the HLG domain by using an HLG curve to obtain the transformed data 10b. The transformed data 10b is also a two-dimensional picture in which a single pixel includes three channels, R, G, and B, and the data of each channel has a preset precision, and the data of each channel is data in the HLG domain. For example, the data of each of the three channels, R, G, and B, of each picture in the two-dimensional picture can be mapped by using an HLG curve to obtain the transformed data 10b.

[0240] S506: Implement conversion from the RGB domain to the YUV domain, that is, convert the converted data 10b obtained in S505 from the RGB domain to the YUV domain to obtain converted data 10c.

[0241] In a possible embodiment, the three channel data of R, G and B of each pixel can be converted into three channel data of Y, U and B in the manner of conversion from RGB color space to YUV 444 color space to obtain converted data 10b. The converted data 10c is a two-dimensional picture in which a single pixel contains three channels of Y, U and V, the data of each channel has a preset precision, and the data of each channel is data in the exponent domain. Each pixel corresponds to the Y channel, the U channel, and the V channel.

[0242] In a possible embodiment, the three channel data of R, G, and B of each pixel can be converted into three channel data of Y, U, and B in the way of conversion from RGB color space to YUV 420 color space to obtain converted data 10c. The converted data 10c is a two-dimensional picture in which a single pixel contains three channels of Y, U, and V, the data of each channel has a preset precision, and the data of each channel is data in the exponent domain. Each pixel corresponds to the Y channel, and four pixels share the U and V channels.

[0243] It should be understood that the conversion from the RGB domain to the YUV domain in S506 may be a conversion from the RGB domain to the XYZ domain or a conversion from the RGB domain to the Lab domain, which is not limited in the present application.

[0244] S507: Perform uniform quantization, that is, perform uniform quantization on the transformed data 10c obtained in S506 to obtain transformed data 12.

[0245] For example, the target quantization precision may be preset, and may be, for example, an 8-bit unsigned integer, a 12-bit unsigned integer, or a 16-bit unsigned integer, which is not limited in this application.

[0246] For example, uniform quantization may be performed on the transformed data 10c based on a target quantization precision to obtain transformed data 12. The transformed data 12 is a two-dimensional picture in which a single pixel includes three channels, Y, U, and V, and the data of each channel has a target quantization precision, and the data of each channel is data in the exponent domain.

[0247] For example, assume that the target quantization precision is a 12-bit unsigned integer and the value range of the converted data 10c is [L1, L2]. [L1, L2] can be divided into 4096 equally spaced values ​​for one of the three channels, Y, U, and V, of a pixel. If the data for that channel of a pixel falls within [z1, z2] (z1 is greater than or equal to L1 and z2 is less than or equal to L2), the data for that channel of the pixel can be quantized as z2. Thus, the converted data 12 is a two-dimensional picture in which a single pixel contains three channels, Y, U, and V, the data for each channel is a 12-bit unsigned integer, and the data for each channel is data in the exponent domain.

[0248] It should be understood that the uniform quantization in S507 may alternatively be replaced with non-uniform quantization or adaptive quantization, which is not a limitation of the present application.

[0249] S508: Pad data into the YUV plane, that is, pad the converted data 12 obtained in S507 into the YUV plane to obtain intermediate data.

[0250] For example, if a conversion from the RGB domain to the YUV domain is performed on the converted data 10b in the manner of a conversion from the RGB color space to the YUV 420 color space, the converted data 12 may be padded into a YUV 420 plane. For example, the Y channel data of each pixel may be padded into a Y plane in the YUV 420 plane, the U channel data of each pixel may be padded into a U plane in the YVU 420 plane, and the V channel data of each pixel may be padded into a V plane in the YUV 420 plane, where the ratio of the Y plane:U plane:V plane is 4:1:1.

[0251] For example, if a conversion from the RGB domain to the YUV domain is performed on the converted data 10b in the manner of a conversion from the RGB color space to the YUV 444 color space, the converted data 12 may be padded into a YUV 444 plane. For example, the Y channel data of each pixel may be padded into a Y plane in the YUV 444 plane, the U channel data of each pixel may be padded into a U plane in the YVU 444 plane, and the V channel data of each pixel may be padded into a V plane in the YUV 444 plane, where the ratio of the Y plane:U plane:V plane is 1:1:1.

[0252] For example, if the data precision of the YUV plane (which may be a YUV 420 plane or a YUV 444 plane) is higher than the data precision of the converted data 12, the most significant bits and least significant bits of the converted data 12 may be padded to convert the precision of the converted data 12 to the same precision as the YUV plane. The padded converted data 12 is then padded into the YUV plane. For example, if the data precision of the YUV plane is 16 bits and the data precision of the converted data 12 is 12 bits, the four most significant bits or the four least significant bits of the data of each channel of each pixel may be padded with 0, and then the padded converted data 12 is padded into the 16-bit YUV plane.

[0253] S509: Perform HEVC encoding, that is, perform HEVC encoding on the intermediate data obtained in S508 to obtain a bitstream corresponding to the illuminance data.

[0254] It should be noted that the order of the six steps S502 to S507 may be changed randomly, which is not limited in the present application.

[0255] FIG. 6 is a schematic diagram of an example encoding procedure. The encoding procedure may include steps of: S601: obtaining data; S602: discarding channels; S603: performing precision conversion; S604: performing min-max normalization; S605: performing linear-to-exponential domain conversion; S606: performing RGB-to-YUV domain conversion; S607: performing uniform quantization; S608: padding data to YUV planes; and S609: performing HEVC encoding. In the embodiment of FIG. 6, the illumination data is represented by spherical harmonic basis coefficients. The transformed data 9 includes transformed data 9a and transformed data 9b. The transformed data 10 includes transformed data 10a, transformed data 10b, and transformed data 10c.

[0256] S601: Obtain data, that is, obtain illuminance data represented by spherical harmonic basis coefficients.

[0257] For example, the illuminance data for each probe may include three channels of R, G, and B or four channels of R, G, B, and A, and the data for each probe includes x spherical harmonic basis coefficients.

[0258] 6, it should be understood that the format of the illuminance data is not limited by the present application. For example, the illuminance data may be R11G11B10F, RGB9E5, or RGBA16F. This is not limited by the present application.

[0259] It should be understood that the illumination data may alternatively be represented by spherical wavelet basis coefficients, although this is not a limitation of this application.

[0260] S602: Discard a channel: If the illuminance data acquired in S601 includes four channels of R, G, B and A, the A channel of the illuminance data is discarded to obtain the converted data 9a.

[0261] For example, the transformed data 9a of each probe may alternatively include three channels of R, G, and B, and the data of each channel includes x spherical harmonic basis coefficients. For example, for S602, see the above description of S502. The details will not be described again here.

[0262] S603: Execute precision conversion, that is, convert the converted data 9a obtained in S602 into a preset precision to obtain converted data 9b.

[0263] For example, the transformed data 9b for each probe may alternatively include three channels of R, G, and B, with each channel of data including x spherical harmonic basis coefficients, each with a preset precision.

[0264] For example, for S603, please refer to the above description of S503, and the details will not be described again here.

[0265] S604: Perform min-max normalization, that is, perform min-max normalization on the transformed data 9b obtained in S603 to obtain transformed data 10a.

[0266] In a possible embodiment, min-max normalization may be performed on spherical harmonic basis coefficients corresponding to the same spherical harmonic basis of the same channel of all probes. For example, the maximum and minimum values ​​of the spherical harmonic basis coefficients corresponding to the i-th spherical harmonic basis (i ranges from 1 to x, where i is an integer) in the R channels of all probes may be determined, and the difference between the maximum and minimum values ​​is calculated to obtain a difference Q3. For the spherical harmonic basis coefficients corresponding to the i-th spherical harmonic basis of the R channel of each probe, the difference between the spherical harmonic basis coefficient corresponding to the i-th spherical harmonic basis of the R channel of that probe and the minimum value is calculated to obtain a difference F5. The difference F5 is divided by the difference Q3 to normalize the spherical harmonic basis coefficient corresponding to the i-th spherical harmonic basis of the R channel of that probe. In this way, the spherical harmonic basis coefficients corresponding to the i-th spherical harmonic basis of the R channels of all probes may be normalized to be within [0, 1]. In this way, the spherical harmonic basis coefficients corresponding to the x spherical harmonic bases of the R channel of all probes can be normalized to be within [0, 1]. Correspondingly, in the same way as normalizing the spherical harmonic basis coefficients corresponding to the x spherical harmonic bases of the R channel of all probes, the spherical harmonic basis coefficients corresponding to the x spherical harmonic bases of the G channel and B channel of all probes can be normalized so that the spherical harmonic basis coefficients corresponding to the x spherical harmonic bases of the G channel and B channel of all probes are normalized to be within [0, 1]. In this way, the transformed data 10a can be obtained. The transformed data 10a of each probe may alternatively include three channels of R, G, and B, and the data of each channel includes x spherical harmonic basis coefficients. The precision of each nine-sided harmonic basis coefficient is a preset precision, and each nine-sided harmonic basis coefficient belongs to [0, 1].

[0267] In a possible aspect, min-max normalization may be performed on spherical harmonic basis coefficients corresponding to the same spherical harmonic basis for the three R, G, and B channels of all probes. For example, the maximum and minimum values ​​of the spherical harmonic basis coefficients corresponding to the i-th spherical harmonic basis in the three R, G, and B channels of all probes may be determined, and the difference between the maximum and minimum values ​​is calculated to obtain a difference Q4. For the spherical harmonic basis coefficients corresponding to the i-th spherical harmonic basis for the R channel of each probe, the difference between the spherical harmonic basis coefficient corresponding to the i-th spherical harmonic basis for the R channel of the probe and the minimum value is calculated to obtain a difference F6. The difference F6 is divided by the difference Q4 to normalize the spherical harmonic basis coefficients corresponding to the i-th spherical harmonic basis for the R channel of the probe. For the spherical harmonic basis coefficients corresponding to the i-th spherical harmonic basis of the G channel of each probe, the difference between the spherical harmonic basis coefficient corresponding to the i-th spherical harmonic basis of the G channel of that probe and the minimum value is calculated to obtain difference F7. Difference F7 is divided by difference Q4 to normalize the spherical harmonic basis coefficient corresponding to the i-th spherical harmonic basis of the G channel of that probe. For the spherical harmonic basis coefficients corresponding to the i-th spherical harmonic basis of the B channel of each probe, the difference between the spherical harmonic basis coefficient corresponding to the i-th spherical harmonic basis of the B channel of that probe and the minimum value is calculated to obtain difference F8. Difference F8 is divided by difference Q4 to normalize the spherical harmonic basis coefficient corresponding to the i-th spherical harmonic basis of the B channel of that probe. In this way, all of the data for the three R, G, and B channels of each probe can be normalized to be within [0, 1] to obtain transformed data 10a.

[0268] S605: Perform a transformation from the linear domain to the exponential domain, that is, transform the transformed data 10a obtained in S604 from the linear domain to the exponential domain to obtain transformed data 10b.

[0269] Alternatively, the transformed data 10b of each probe may include three channels of R, G, and B, and the data of each channel includes x spherical harmonic basis coefficients, each of which has a preset precision and belongs to an exponent domain.

[0270] For example, for S605, please refer to the above description of S505, and the details will not be described again here.

[0271] S606: Implement conversion from the RGB domain to the YUV domain, that is, convert the converted data 10b obtained in S605 from the RGB domain to the YUV domain to obtain converted data 10c.

[0272] Alternatively, the transformed data 10c of each probe may include three channels: Y, U, and V, and the data of each channel includes x spherical harmonic basis coefficients, each of which has a preset precision and belongs to an exponent domain.

[0273] For example, for S606, please refer to the above description of S506, and the details will not be described again here.

[0274] S607: Perform uniform quantization, that is, perform uniform quantization on the transformed data 10c obtained in S606 to obtain transformed data 12.

[0275] Alternatively, the transformed data 12 of each probe may include three channels: Y, U, and V, and the data of each channel includes x spherical harmonic basis coefficients, each of which has a precision equal to the target quantization precision and belongs to an exponent domain.

[0276] For example, for S607, please refer to the above description of S507, and the details will not be described again here.

[0277] S608: Pad data into the YUV plane, that is, pad the converted data 12 obtained in S607 into the YUV plane to obtain intermediate data.

[0278] S609: Perform HEVC encoding, that is, perform HEVC encoding on the intermediate data obtained in S608 to obtain a bitstream corresponding to the illuminance data.

[0279] For example, for S608 and S609, please refer to the above description of S508 and S509, and the details will not be described again here.

[0280] It should be noted that the order of the six steps S602 to S607 may be changed randomly, which is not limited in the present application.

[0281] FIG. 7 is a schematic diagram of an example encoding procedure. The encoding procedure may include steps of: S701: obtaining data; S702: discarding channels; S703: performing precision conversion; S704: performing adaptive normalization; S705: performing a linear domain to exponential domain conversion; S706: performing an RGB domain to YUV domain conversion; S707: performing a picture domain to transform domain conversion; S708: performing uniform quantization; S709: padding data to YUV planes; and S710: performing HEVC encoding. In the embodiment of FIG. 7, the illumination data is represented by a two-dimensional picture. The transformed data 9 includes transformed data 9a and transformed data 9b. The transformed data 10 includes transformed data 10a, transformed data 10b, transformed data 10c, and transformed data 10d.

[0282] S701: Obtain data, that is, obtain illumination data represented by a two-dimensional picture.

[0283] S702: Discard a channel: If the illuminance data acquired in S701 includes four channels of R, G, B and A, the A channel of the illuminance data is discarded to obtain the converted data 9a.

[0284] S703: Perform precision conversion, that is, convert the converted data 9a obtained in S702 into a preset precision to obtain converted data 9b.

[0285] S704: Perform adaptive normalization, that is, perform adaptive normalization on the transformed data 9b obtained in S703 to obtain transformed data 10a.

[0286] S705: Perform a transformation from the linear domain to the exponential domain, that is, transform the transformed data 10a obtained in S704 from the linear domain to the exponential domain to obtain transformed data 10b.

[0287] S706: Perform conversion from the RGB domain to the YUV domain, that is, convert the converted data 10b obtained in S705 from the RGB domain to the YUV domain to obtain converted data 10c.

[0288] For example, for S701-S701, please refer to the above description of S501-S506, and the details will not be described again here.

[0289] S707: Perform conversion from the picture domain to the transform domain, that is, convert the transformed data 10c obtained in S706 from the picture domain to the transform domain to obtain transformed data 10d.

[0290] In a possible manner, the transformed data 10c represented by a two-dimensional picture may be decomposed to obtain spherical harmonic bases and spherical harmonic basis coefficients, which may be used as the transformed data 10d.

[0291] In a possible manner, the transformed data 10c represented by a two-dimensional picture may be decomposed to obtain eigenvectors and eigenvector coefficients, which may be used as transformed data 10d.

[0292] In a possible manner, the transformed data 10c represented by a two-dimensional picture may be decomposed to obtain spherical wavelet bases and spherical wavelet base coefficients, which may be used as the transformed data 10d.

[0293] S708: Perform uniform quantization, that is, perform uniform quantization on the transformed data 10d obtained in S707 to obtain transformed data 12.

[0294] S709: Pad data into the YUV plane, that is, pad the converted data 12 obtained in S708 into the YUV plane to obtain intermediate data.

[0295] S710: Perform HEVC encoding, that is, perform HEVC encoding on the intermediate data obtained in S709 to obtain a bitstream corresponding to the illuminance data.

[0296] For example, for S708 and S710, please refer to the above description of S507-S509, and the details will not be described again here.

[0297] It should be noted that the order of the seven steps S702 to S708 may be changed randomly, which is not limited in the present application. If S707 is performed before S704, S704 can be replaced by S604.

[0298] The following describes in detail the process of encoding visibility data by using the encoding procedure shown in Figure 4c(5) as an example.

[0299] 8 is a schematic diagram of an example of an encoding procedure. The encoding procedure may include steps S801: obtaining data, S802: performing channel splitting, S803a: performing precision conversion, S803b: performing precision conversion, S804a: performing fixed parameter normalization, S804b: performing fixed parameter normalization, S805a: performing uniform quantization, S805b: performing uniform quantization, S806a: padding data into YUV planes, S806b: padding data into YUV planes, S807a: performing HEVC encoding, and S807b: performing HEVC encoding. In the embodiment of FIG. 8, the visibility data is represented by a two-dimensional picture. The transformed data 9 includes transformed data 9a1, transformed data 9a2, transformed data 9b1, and transformed data 9b2. The transformed data 10 includes transformed data 10a and transformed data 10b. The transformed data 12 includes transformed data 12a and transformed data 12b. The intermediate data includes intermediate data 1 and intermediate data 2.

[0300] S801: Obtain data, that is, obtain visibility data, which is a two-dimensional picture in which a single pixel includes two channels of R and G.

[0301] For example, the visibility data of the K probes may be a two-dimensional picture in which a single picture includes two channels, R and G. In a possible embodiment, the R channel corresponds to distance data, and the G channel corresponds to the square of the distance data. In a possible embodiment, the G channel corresponds to distance data, and the R channel corresponds to the square of the distance data. Each probe corresponds to multiple pixels in the two-dimensional picture.

[0302] S802: Perform channel splitting, that is, perform channel splitting on the visibility data obtained in S801 to obtain transformed data 9a1 and transformed data 9a2.

[0303] For example, channel splitting may be performed on the visibility data, where the visibility data is separated into two independent channels, and the data corresponding to the R channel obtained by channel splitting may be referred to as transformed data 9a1, and the data corresponding to the G channel obtained by channel splitting may be referred to as transformed data 9a2.

[0304] S803a: Perform precision conversion, that is, convert the converted data 9a1 obtained in S802 into a preset precision to obtain converted data 9b1.

[0305] S803b: Perform precision conversion, that is, convert the converted data 9a2 obtained in S802 into a preset precision to obtain converted data 9b2.

[0306] For example, for S803a and S803b, please refer to the above description of S503, and the details will not be described again here.

[0307] It should be noted that S803a and S803b may be performed in parallel or sequentially, which is not a limitation in the present application.

[0308] S804a: Perform fixed parameter normalization, that is, perform fixed parameter normalization on the transformed data 9b1 obtained in S803a to obtain transformed data 10a.

[0309] For example, a preset fixed parameter may be set based on the maximum distance value. The maximum distance value may be set based on requirements, for example, 1.5 times the probe spacing in three-dimensional space. This is not limited in the present application. For specific normalization process, please refer to the above description of S504. The details will not be described again here.

[0310] For example, data obtained by performing fixed parameter normalization on transformed data 9b1 may be referred to as transformed data 10a, where transformed data 10a belongs to [0,1].

[0311] It should be noted that if the data of a channel of a pixel is greater than the normalization parameter, the data of that channel of the pixel may be truncated so that the data of that channel of the pixel is equal to the normalization parameter. The data of that channel of the pixel is then divided by the normalization parameter to normalize the data of that channel of the pixel. In this case, the data of that channel of the pixel is normalized to 1.

[0312] S804b: Perform fixed parameter normalization, that is, perform fixed parameter normalization on the transformed data 9b2 obtained in S803b to obtain transformed data 10b.

[0313] For example, the square of the maximum distance value may be preset as the normalization parameter. Then, maximum normalization may be performed on the transformed data 9b2, with reference to S804a, to obtain transformed data 10b. Details will not be described again here.

[0314] It should be noted that S804a and S804b may be performed in parallel or sequentially, which is not a limitation of the present application.

[0315] S805a: Perform uniform quantization, that is, perform uniform quantization on the transformed data 10a obtained in S804a to obtain transformed data 12a.

[0316] S805b: Perform uniform quantization, that is, perform uniform quantization on the transformed data 10b obtained in S804b to obtain transformed data 12b.

[0317] For example, for S805a and S805b, please refer to the above description of S507, and the details will not be described again here.

[0318] It should be noted that S805a and S805b may be performed in parallel or sequentially, which is not a limitation in the present application.

[0319] S806a: Pad data into a YUV plane, that is, pad the converted data 12a obtained in S805a into a YUV plane to obtain intermediate data 1.

[0320] S806b: Pad data into the YUV plane, that is, pad the converted data 12b obtained in S805b into the YUV plane to obtain intermediate data 2.

[0321] For example, for S806a and S806b, please refer to the above description of S508, and the details will not be described again here.

[0322] It should be noted that S806a and S806b may be performed in parallel or sequentially, which is not a limitation of the present application.

[0323] S807a: Perform HEVC encoding, that is, perform HEVC encoding on the intermediate data 1 obtained in S806a to obtain a corresponding bitstream.

[0324] S807b: Perform HEVC encoding, that is, perform HEVC encoding on the intermediate data 2 obtained in S806b to obtain a corresponding bitstream.

[0325] For example, a bitstream obtained by encoding intermediate data 1 and a bitstream obtained by encoding intermediate data 2 may be called bitstreams corresponding to visibility data.

[0326] It should be noted that S807a and S807b may be performed in parallel or sequentially, which is not a limitation in the present application.

[0327] 9 is a schematic diagram of an example of an encoding procedure. The encoding procedure may include steps S901: obtaining data, S902: performing channel splitting, S903a: performing precision conversion, S903b: performing precision conversion, S904a: performing min-max normalization, S904b: performing min-max normalization, S905a: performing uniform quantization, S905b: performing uniform quantization, S906a: padding data into YUV planes, S906b: padding data into YUV planes, S907a: performing HEVC encoding, and S907b: performing HEVC encoding. In the embodiment of FIG. 9, the visibility data is represented by spherical harmonic basis coefficients. The transformed data 9 includes transformed data 9a1, transformed data 9a2, transformed data 9b1, and transformed data 9b2. The transformed data 10 includes transformed data 10a and transformed data 10b. The transformed data 12 includes transformed data 12a and transformed data 12b. The intermediate data may include intermediate data 1 and intermediate data 2.

[0328] S901: Data is acquired, that is, visibility data including two channels of R and G, each channel being represented by spherical harmonic basis coefficients.

[0329] For example, the visibility data for each probe includes two channels, R and G, and the data for each channel includes x spherical harmonic basis coefficients. In a possible embodiment, the R channel corresponds to the distance data and the G channel corresponds to the square of the distance data. In a possible embodiment, the G channel corresponds to the distance data and the R channel corresponds to the square of the distance data.

[0330] S902: Perform channel splitting, that is, perform channel splitting on the visibility data obtained in S901 to obtain transformed data 9a1 and transformed data 9a2.

[0331] S903a: Perform precision conversion, that is, convert the converted data 9a1 obtained in S902 into a preset precision to obtain converted data 9b1.

[0332] S903b: Perform precision conversion, that is, convert the converted data 9a2 obtained in S902 into a preset precision to obtain converted data 9b2.

[0333] For example, for S903a and S903b, please refer to the above description of S503, and the details will not be described again here.

[0334] It should be noted that S903a and S903b may be performed in parallel or sequentially, which is not a limitation in the present application.

[0335] S904a: Perform min-max normalization, that is, perform min-max normalization on the transformed data 9b1 obtained in S903a to obtain transformed data 10a.

[0336] S904b: Perform min-max normalization, that is, perform fixed parameter normalization on the transformed data 9b2 obtained in S903b to obtain transformed data 10b.

[0337] For example, for S904a and S904b, please refer to the above description of S604, and the details will not be described again here.

[0338] It should be noted that S904a and S904b may be performed in parallel or sequentially, which is not a limitation of the present application.

[0339] S905a: Perform uniform quantization, that is, perform uniform quantization on the transformed data 10a obtained in S904a to obtain transformed data 12a.

[0340] S905b: Perform uniform quantization, that is, perform uniform quantization on the transformed data 10b obtained in S904b to obtain transformed data 12b.

[0341] For example, for S905a and S905b, please refer to the above description of S507, and the details will not be described again here.

[0342] It should be noted that S905a and S905b may be performed in parallel or sequentially, which is not a limitation in the present application.

[0343] S906a: Pad data into a YUV plane, that is, pad the converted data 12a obtained in S905a into a YUV plane to obtain intermediate data 1.

[0344] S906b: Pad data into the YUV plane, that is, pad the converted data 12b obtained in S905b into the YUV plane to obtain intermediate data 2.

[0345] For example, for S906a and S906b, please refer to the above description of S508, and the details will not be described again here.

[0346] It should be noted that S906a and S906b may be performed in parallel or sequentially, which is not a limitation of the present application.

[0347] S907a: Perform HEVC encoding, that is, perform HEVC encoding on the intermediate data 1 obtained in S906a to obtain a corresponding bitstream.

[0348] S907b: Perform HEVC encoding, that is, perform HEVC encoding on the intermediate data 2 obtained in S906b to obtain a corresponding bitstream.

[0349] For example, a bitstream obtained by encoding intermediate data 1 and a bitstream obtained by encoding intermediate data 2 may be called bitstreams corresponding to visibility data.

[0350] It should be noted that S907a and S907b may be performed in parallel or sequentially, which is not a limitation in the present application.

[0351] 10 is a schematic diagram of an example of an encoding procedure. The encoding procedure may include: S1001: obtaining data; S1002: performing channel splitting; S1003a: performing precision conversion; S1003b: performing precision conversion; S1004a: performing fixed parameter normalization; S1004b: performing fixed parameter normalization; S1005a: performing a conversion from a picture domain to a transform domain; S1005b: performing a conversion from a picture domain to a transform domain; S1006a: performing uniform quantization; S1006b: performing uniform quantization; S1007a: padding data into a YUV plane; S1007b: padding data into a YUV plane; S1008a: performing HEVC encoding; and S1008b: performing HEVC encoding. In the embodiment of Fig. 10, the visibility data is represented by a two-dimensional picture. The transformed data 9 includes transformed data 9a1, transformed data 9a2, transformed data 9b1, and transformed data 9b2. The transformed data 10 includes transformed data 10a1, transformed data 10a2, transformed data 10b1, and transformed data 10b2. The transformed data 12 includes transformed data 12a and transformed data 12b. The intermediate data includes intermediate data 1 and intermediate data 2.

[0352] S1001: Data is acquired, that is, visibility data is acquired, which is a two-dimensional picture in which a single pixel includes two channels, R and G.

[0353] S1002: Perform channel division, that is, perform channel division on the visibility data obtained in S1001 to obtain transformed data 9a1 and transformed data 9a2.

[0354] S1003a: Execute precision conversion, that is, convert the converted data 9a1 obtained in S1002 into a preset precision to obtain converted data 9b1.

[0355] S1003b: Execute precision conversion, that is, convert the converted data 9a2 obtained in S1002 into a preset precision to obtain converted data 9b2.

[0356] S1004a: Perform fixed parameter normalization, that is, perform fixed parameter normalization on the transformed data 9b1 obtained in S1003a to obtain transformed data 10a1.

[0357] S1004b: Perform fixed parameter normalization, that is, perform fixed parameter normalization on the transformed data 9b2 obtained in S1003b to obtain transformed data 10a2.

[0358] For example, for S1001, S1002, S1003a, S1003b, S1004a, and S1004b, please refer to the above descriptions of S801, S802, S803a, S803b, S804a, and S804b, and the details will not be described again here.

[0359] S1005a: Perform a conversion from the picture domain to the transform domain, that is, convert the transformed data 10a1 obtained in S1004a from the picture domain to the transform domain to obtain transformed data 10b1.

[0360] S1005 b : Implement a conversion from the picture domain to the conversion domain, that is, convert the converted data 10a2 obtained in S1004b from the picture domain to the conversion domain to obtain converted data 10b2.

[0361] For example, see the above description of S707 for S1005a and S1005b, which is not limited in this application.

[0362] It should be noted that S1005a and S1005b may be performed in parallel or sequentially, which is not a limitation in the present application.

[0363] S1006a: Perform uniform quantization, that is, perform uniform quantization on the transformed data 10b1 obtained in S1005a to obtain transformed data 12a.

[0364] S1006b: Perform uniform quantization, that is, perform uniform quantization on the transformed data 10b2 obtained in S1005b to obtain transformed data 12b.

[0365] For example, for S1006a and S1006b, please refer to the above description of S507, and the details will not be described again here.

[0366] It should be noted that S1006a and S1006b may be performed in parallel or sequentially, which is not a limitation of the present application.

[0367] S1007a: Pad data into the YUV plane, that is, pad the converted data 12a obtained in S1006a into the YUV plane to obtain intermediate data 1.

[0368] S1007b: Pad data into the YUV plane, that is, pad the converted data 12b obtained in S1006b into the YUV plane to obtain intermediate data 2.

[0369] For example, for S1007a and S1007b, please refer to the above description of S508, and the details will not be described again here.

[0370] It should be noted that S1007a and S1007b may be performed in parallel or sequentially, which is not a limitation in the present application.

[0371] S1008a: Perform HEVC encoding, that is, perform HEVC encoding on the intermediate data 1 obtained in S1007a to obtain a corresponding bitstream.

[0372] S1008b: Perform HEVC encoding, that is, perform HEVC encoding on the intermediate data 2 obtained in S1007b to obtain a corresponding bitstream.

[0373] For example, a bitstream obtained by encoding intermediate data 1 and a bitstream obtained by encoding intermediate data 2 may be called bitstreams corresponding to visibility data.

[0374] It should be noted that S1008a and S1008b may be performed in parallel or sequentially, which is not a limitation of the present application.

[0375] It should be noted that S1008a and S1008b may be coded by using two coding modules (S807a and S807b may also be coded by using two coding modules, and S907a and S907b may also be coded by using two coding modules). The similarity between the distance data and the similarity between the squares of the distance data are higher than the similarity between the distance data and the squares of the distance data. In this way, it is easier to use two coding modules to separately perform inter-frame coding on the distance data and on the squares of the distance data, thereby further reducing the bit rate under the same rendering effect.

[0376] It should be noted that channel splitting may be performed on the visibility data after the precision conversion step, normalization step, and quantization step are performed. Furthermore, S1002 (S802 or S902) may be performed, that is, the precision conversion step, normalization step, and quantization step are performed directly on the visibility data including two channels of R and G. Then, the data of the two channels of R and G in the visibility data are padded into the same YUV plane, and the intermediate data is encoded. This is not a limitation of the present application.

[0377] The following describes in detail the process of encoding probe data by using the encoding procedure shown in Figure 4c(2) as an example.

[0378] 11 is a schematic diagram of an example encoding procedure. The encoding procedure may include steps of: S1101: obtaining data; S1102: performing dimensional transformation; S1103: performing PCA decomposition; S1104: performing min-max normalization; S1105: performing uniform quantization; and S1106: performing entropy coding. In the embodiment of FIG. 11, the probe data may include illuminance data and / or visibility data. The transformed data 10 may include transformed data 10a and transformed data 10b.

[0379] S1101: Data is acquired, that is, probe data is acquired.

[0380] For example, in the embodiment of Figure 11, the probe data includes illumination data and / or visibility data. The illumination data can be represented by a two-dimensional picture or spherical harmonic basis (or spherical wavelet basis) coefficients. The visibility data can be represented by a two-dimensional picture or spherical harmonic basis (or spherical wavelet basis) coefficients. This is not a limitation of the present application.

[0381] S1102: Perform a dimension transformation, that is, perform a dimension transformation on the probe data acquired in S1101 to obtain a two-dimensional matrix.

[0382] For example, if the illumination data is represented by a two-dimensional picture, for a pixel in the two-dimensional picture, the R channel data of that pixel may be used as an element of a two-dimensional matrix, the G channel data of that pixel may be used as an element of a two-dimensional matrix, and the B channel data of that pixel may be used as an element of a two-dimensional matrix.

[0383] For example, if illumination data is represented by spherical harmonic basis coefficients (or spherical wavelet basis coefficients), for each probe, the spherical harmonic basis (or spherical wavelet basis) coefficients of each spherical harmonic basis (or spherical wavelet basis) of the R channel of the probe may be used as elements of a two-dimensional matrix, i.e., n elements may be obtained. The spherical harmonic basis (or spherical wavelet basis) coefficients of each spherical harmonic basis (or spherical wavelet basis) of the G channel of the probe may be used as elements of a two-dimensional matrix, again, n elements may be obtained. The spherical harmonic basis (or spherical wavelet basis) coefficients of each spherical harmonic basis (or spherical wavelet basis) of the B channel of the probe may be used as elements of a two-dimensional matrix, again, n elements may be obtained. In this way, for a given probe, the illumination data of the probe may be expanded to 3n elements of a two-dimensional matrix.

[0384] For example, the spherical harmonic basis coefficients of the same spherical harmonic basis in all the probes may be arranged in one row or one column. For example, the spherical harmonic basis coefficients of the same spherical harmonic basis in the same channel in all the probes may also be arranged in one row or one column. This is not a limitation of the present application.

[0385] For example, the method for converting visibility data into a two-dimensional matrix is ​​similar to the method for converting illuminance data into a two-dimensional matrix, and the details will not be described again here.

[0386] For example, illuminance data may be converted into a two-dimensional matrix, and visibility data may be converted into another two-dimensional matrix.

[0387] It should be noted that in the process of converting the probe data into a two-dimensional matrix, if the probe data cannot be directly converted into a two-dimensional matrix of a preset size, the missing part may be filled by using invalid data. The preset size may be set based on requirements, which is not limited in the present application.

[0388] For example, the obtained two-dimensional matrix is ​​the transformed data 9 above.

[0389] It should be noted that matrices of other dimensions (e.g., five-dimensional matrices) can also be obtained by performing a dimensional transformation on the probe data, which is not limited in this application. For illustration purposes, the following uses an example in which a dimensional transformation is performed on the probe data to obtain a two-dimensional matrix.

[0390] S1103: Perform PCA decomposition, that is, perform PCA decomposition on the two-dimensional matrix obtained in S1102 to obtain transformed data 10a.

[0391] For example, a PCA decomposition can be performed on a two-dimensional matrix to divide it into eigenvectors and eigenvector coefficients, which are then used as the transformed data 10a.

[0392] S1104: Perform min-max normalization, that is, perform min-max normalization on the transformed data 10a obtained in S1103 to obtain transformed data 10b.

[0393] For example, min-max normalization may be performed separately on the eigenvectors and eigenvector coefficients of the transformed data 10a. For details, see the above description of min-max normalization. The details will not be repeated here.

[0394] S1105: Perform uniform quantization, that is, perform uniform quantization on the transformed data 10b obtained in S1104 to obtain intermediate data.

[0395] For example, uniform quantization may be performed separately on the eigenvectors and eigenvector coefficients of the transformed data 10b. For details, see the above description of uniform quantization in S507. The details will not be described again here.

[0396] S1106: Perform entropy encoding, that is, perform entropy encoding on the intermediate data obtained in S1105 to obtain a corresponding bitstream.

[0397] For example, if the probe data is illuminance data, entropy encoding may be performed on the illuminance data to obtain a bitstream corresponding to the illuminance data.

[0398] For example, if the probe data is visibility data, entropy coding may be performed on the visibility data to obtain a bitstream corresponding to the visibility data.

[0399] For example, if the probe data is illuminance data and visibility data, entropy encoding may be performed on the illuminance data to obtain a bitstream corresponding to the illuminance data, and entropy encoding may be performed on the visibility data to obtain a bitstream corresponding to the visibility data.

[0400] For example, load balancing may be performed based on the amount of probe data to determine a target bit rate and encoding scheme for the probe data.

[0401] In a possible aspect, the first target bit rate corresponding to the illuminance data and the second target bit rate corresponding to the visibility data may be determined based on the amount of illuminance data, the amount of visibility data, and a preset bit rate. In this way, to obtain the bit rates, the intermediate data corresponding to the illuminance data may be coded based on the first target bit rate, and the intermediate data corresponding to the visibility data may be coded based on the second target bit rate. The preset bit rates may be set based on requirements, which is not limited in the present application.

[0402] For example, the first target bit rate and the second target bit rate may be determined based on the ratio of the amount of illuminance data to the amount of visibility data. Assume that the preset bit rate is 1000 kbps. If the ratio of the amount of visibility data to the amount of illuminance data is 9:1, the first target bit rate may be determined to be 900 kbps and the second target bit rate may be determined to be 100 kbps.

[0403] In this way, appropriate bit rates can be allocated to the illuminance data and the visibility data, and compared with the case where the ratio of the bit stream corresponding to the illuminance data to the bit stream corresponding to the visibility data is fixed, in this application, the rendering effect can be improved when the bit streams are the same.

[0404] In a possible aspect, the coding scheme corresponding to the illuminance data and the coding scheme corresponding to the visibility data are determined based on the amount of illuminance data, the amount of visibility data, and channel feedback information. The coding scheme includes intra-frame coding or inter-frame coding. In this way, the intermediate data corresponding to the illuminance data may then be coded using the coding scheme corresponding to the illuminance data, and the intermediate data corresponding to the visibility data may then be coded using the coding scheme corresponding to the visibility data to obtain a bitstream. In this way, it is possible to balance the bitstream size and the rendering effect.

[0405] For example, the bit rate that can be carried by the channel can be determined based on the channel feedback information, and then whether intra-frame coding or inter-frame coding is performed on the illuminance data is determined based on the amount of illuminance data and the bit rate that can be carried by the channel. For example, a predicted bit rate of the corresponding bit stream after intra-frame coding is performed on the illuminance data can be estimated. If the predicted bit rate is less than or equal to the bit rate that can be carried by the channel, intra-frame coding may be performed on the illuminance data. prediction The bit rate that the channel can carry Bitrate If the illuminance data is greater than 1, inter-frame coding may be performed on the illuminance data. Similarly, whether intra-frame coding or inter-frame coding is performed on the visibility data may also be determined in the above manner, which is not limited in this application.

[0406] For example, first attribute data used to perform data format conversion on the probe data (e.g., normalization parameters (parameters used for normalization), quantization parameters (parameters used for quantization), color space conversion parameters (parameters used for conversion from RGB domain to YUV / XYZ / Lab domain), exponential conversion parameters (parameters used for conversion from linear domain to exponential domain), PQ conversion parameters (parameters used for conversion from linear domain to PQ domain, e.g., parameters related to a PQ curve), HLG conversion parameters (parameters used for conversion from linear domain to HLG domain, e.g., parameters related to an HLG curve), etc.) may also be sent to the second device, so that the second device performs data format conversion based on the first attribute data to recover the probe data.

[0407] For example, second attribute data used in the rendering process may also be sent to the second device, so that the second device performs rendering based on the second attribute data.

[0408] In a possible aspect, the attribute data of the K probes may be coded to obtain a bitstream corresponding to the attribute data. For example, the coding method for the attribute data of the K probes may be context coding or other coding methods, which are not limited in the present application.

[0409] In a possible embodiment, a third manner rearrangement may be performed on the attribute data of the K probes, and then the rearranged attribute data is encoded into a corresponding bitstream. The third manner rearrangement may be used for concatenation, i.e., the first attribute data and the second attribute data may be concatenated, and then the concatenated data is encoded to obtain a corresponding bitstream.

[0410] In a possible manner, if the first device encodes only illuminance data, the obtained bitstream includes intermediate data corresponding to the illuminance data.

[0411] In a possible manner, if the first device encodes only visibility data, the obtained bitstream includes intermediate data corresponding to the visibility data.

[0412] In a possible aspect, when the first device encodes both illuminance data and visibility data, the obtained bitstream may include intermediate data corresponding to the visibility data and intermediate data corresponding to the illuminance data.

[0413] In a possible aspect, when the first device encodes the illuminance data and the attribute data, the obtained bitstream may include intermediate data corresponding to the illuminance data and the attribute data.

[0414] In a possible aspect, when the first device encodes only the visibility data and the attribute data, the obtained bitstream may include intermediate data corresponding to the visibility data and the attribute data.

[0415] In a possible aspect, when the first device encodes illuminance data, visibility data, and attribute data, the obtained bitstream may include intermediate data corresponding to the visibility data, intermediate data corresponding to the illuminance data, and attribute data.

[0416] For example, after a bitstream is acquired, bitstream structure information may be generated based on the bitstream structure, and then the bitstream structure information is written into the acquired bitstream. For example, the bitstream structure may include, but is not limited to, the number of probes, the position, length, and data format of intermediate data corresponding to illuminance data, the position, length, and data format of intermediate data corresponding to visibility data, the execution order of data format conversion types corresponding to visibility data, the position, length, and data format of first attribute data, the position, length, and data format of second attribute data, etc. This is not limited in the present application.

[0417] The data format conversion types may include various first-way rearrangement types (e.g., discarding data of some channels, channel splitting, precision conversion, and dimension conversion), various normalization types (e.g., adaptive normalization, fixed parameter normalization, min-max normalization, and z-score normalization), various domain conversion types (e.g., conversion from linear to nonlinear domain, conversion from RGB domain to YUV domain, conversion from RGB domain to XYZ domain, conversion from RGB domain to Lab domain, and conversion from picture domain to transform domain), various quantization types (e.g., uniform quantization and non-uniform quantization), second-way rearrangement types such as padding data YUV planes, etc. This is not limited in the present application.

[0418] It should be understood that the bitstream structure information may include more or less information than the above information, and this is not limited in this application. Also, the bitstream may alternatively not include bitstream structure information, which can be specifically set based on requirements, and this is also not limited in this application.

[0419] It should be understood that the locations of the intermediate data corresponding to visibility data, the intermediate data corresponding to illuminance data, the attribute data, and the bitstream structure information in the bitstream are not limited by this application.

[0420] Fig. 12 is a diagram of an example of the structure of a bitstream. The bitstream shown in Fig. 12 includes intermediate data corresponding to visibility data, intermediate data corresponding to illuminance data, attribute data, and bitstream structure information.

[0421] For example, after obtaining a bitstream corresponding to the probe data, the first device may send the probe data to the second device, so that the second device can determine shading effects for objects in a three-dimensional scene based on the probe data in a rendering process.

[0422] 13 is a diagram of an example of an electronic device. In the embodiment of FIG. 13, the electronic device is a first device, which can be configured to perform the method of the above embodiment. Therefore, for the advantageous effects that the electronic device can achieve, please refer to the advantageous effects of the corresponding method given above. The details will not be described again here. The first device: a data acquisition module 1301 configured to acquire probe data corresponding to one or more probes in the three-dimensional scene, the probe data being used by the second device to determine shading effects of objects in the three-dimensional scene in a rendering process; a data format conversion module 1302 configured to perform a data format conversion on the probe data to obtain intermediate data, the data format conversion including a domain conversion; an encoding module 1303 configured to encode the intermediate data to obtain a corresponding bitstream; may include:

[0423] For example, the data format conversion module 1302 a domain transformation module 13021 configured to perform a first operation on the probe data to obtain transformed data, the first operation being a domain transformation, or to perform a second operation on the transformed data to obtain intermediate data, the second operation being a domain transformation; a quantization module 13022 configured to perform a first operation on the probe data to obtain transformed data, the first operation being quantization, or to perform a second operation on the transformed data to obtain intermediate data, the second operation being quantization; a rearrangement module 13023 configured to perform a first operation on the probe data to obtain transformed data, the first operation being a first modality rearrangement, or to perform a second operation on the transformed data to obtain intermediate data, the second operation being a first modality rearrangement; Includes:

[0424] For example, the domain transformation module 13021 is further configured to perform a third operation on the probe data before performing the first operation on the probe data, the third operation being a domain transformation.

[0425] For example, the quantization module 13022 is further configured to perform a third operation on the probe data before performing the first operation on the probe data, the third operation being quantization.

[0426] For example, the rearrangement module 13023 is further configured to perform a third operation on the probe data before performing the first operation on the probe data, the third operation being a first manner rearrangement.

[0427] For example, the rearrangement module is further configured, after performing the second processing on the transformed data and before obtaining the intermediate data, to perform a second-style rearrangement on the data obtained by the second processing to obtain the intermediate data, where the second-style rearrangement is padding the data obtained by the second processing into a YUV plane.

[0428] For example, when the probe data is illuminance data, and the illuminance data includes multiple channels, the first modality rearrangement includes at least one of the following: discarding data of some channels, converting the illuminance data to a preset precision, or performing a dimensional transformation on the illuminance data.

[0429] For example, when the probe data is visibility data and the visibility data includes multiple channels, the first modality rearrangement includes at least one of the following: performing channel splitting, converting the visibility data to a preset precision, or performing a dimensional transformation on the visibility data.

[0430] For example, the domain transformation includes at least one of the following: a transformation from a non-normalized domain to a normalized domain, a transformation from a linear domain to a non-linear domain, a transformation from an RGB domain to a YUV domain, a transformation from an RGB domain to an XYZ domain, a transformation from an RGB domain to an Lab domain, and a transformation from a picture domain to a transform domain.

[0431] For example, the probe data may be represented by a two-dimensional picture, spherical harmonic basis coefficients, or spherical wavelet basis coefficients.

[0432] For example, the encoding module 1303 is further configured to encode attribute data of the one or more probes to obtain a bitstream, the attribute data including first attribute data for data format conversion and / or second attribute data used in the rendering process.

[0433] For example, if the probe data includes illuminance data and visibility data, the bitstream includes bitstream structure information, which includes the location of intermediate data corresponding to the illuminance data and / or the location of intermediate data corresponding to the visibility data.

[0434] For example, the first device may If the probe data includes illuminance data and visibility data, the device further includes a bitstream load balancing module 1304 configured to determine a first target bit rate corresponding to the illuminance data and a second target bit rate corresponding to the visibility data based on the amount of illuminance data, the amount of visibility data, and a preset bit rate.

[0435] The encoding module 1303 is particularly configured to encode the intermediate data corresponding to the illuminance data based on a first target bit rate, and encode the intermediate data corresponding to the visibility data based on a second target bit rate to obtain a bit rate.

[0436] For example, if the probe data includes illuminance data and visibility data, the bitstream load balancing module 1304 is further configured to determine, based on the amount of illuminance data, the amount of visibility data, and channel feedback information, an encoding scheme corresponding to the illuminance data and an encoding scheme corresponding to the visibility data, where the encoding scheme includes intra-frame encoding or inter-frame encoding.

[0437] The encoding module 1303 is particularly configured to encode the intermediate data corresponding to the illuminance data by using an encoding scheme corresponding to the illuminance data, and encode the intermediate data corresponding to the visibility data by using an encoding scheme corresponding to the visibility data, to obtain a bitstream.

[0438] FIG. 14a is a diagram of an example of compression effect. FIG. 14a shows a PSNR (peak signal to noise ratio) curve corresponding to lossless compression and a PSNR curve corresponding to the encoding method of the present application. The PSNR may represent the difference between an unprocessed picture (referring to a picture obtained before compression is performed on the encoder side in FIG. 14a) and a processed picture (referring to a picture obtained on the decoder side by decoding the bitstream sent by the encoder side in FIG. 14a). The larger the PSNR, the smaller the loss of the processed picture compared to the unprocessed picture. The PSNR can be used to measure the quality or rendering effect of a picture obtained by compression and decompression. In FIG. 14a, the ordinate is PSNR, and the abscissa is frame number.

[0439] See Figure 14a. For example, the curve corresponding to the reference (delay 1 frame) is a PSNR curve corresponding to lossless compression of the prior art when one frame is delayed, with an average PSNR of 58.49 dB. The curve corresponding to the probe compression (delay 1 frame) is a PSNR curve corresponding to lossy compression performed by the encoding method of the present application when one frame is delayed, with an average PSNR of 50.59 dB. The curve corresponding to the reference (delay 4 frame) is a PSNR curve corresponding to lossless compression of the prior art when four frames are delayed, with an average PSNR of 51.15 dB. The curve corresponding to the probe compression (delay 4 frame) is a PSNR curve corresponding to lossy compression performed by the encoding method of the present application when four frames are delayed, with an average PSNR of 47.97 dB.

[0440] It should be noted that for two pictures with PSNR greater than 40 dB, the difference perceived by the user between the two pictures is unclear, and it can be seen that the rendering effect is not significantly reduced by the encoding method of the present application.

[0441] In the prior art, if lossy compression is performed after data format conversion, the accuracy of the probe data obtained by decoding by the decoder side will be significantly reduced, and the rendering effect will then be significantly reduced. After the data format conversion process provided in the present application is used, both lossless and lossy compression can be performed on the processed intermediate data. As shown in FIG. 14a, even if lossy compression is performed on the intermediate data obtained by the data format conversion process of the present application, the decoder side can still obtain high-quality pictures, and the user can barely notice the difference in rendering effect with the naked eye. However, in the method provided in the present application, after data format conversion and lossy compression are performed on the probe data, the amount of data is significantly reduced, the bit rate can be significantly reduced, and the transmission data can be reduced, thereby reducing the rendering delay of the client and improving the user experience, especially in scenarios with high requirements for real-time performance.

[0442] Figure 14b is an illustration of an example of the compression effect, where the ordinate is the amount of data in kb and the abscissa is frames.

[0443] See Figure 14b. For example, the probe data before encoding is more than 40,000 KB. The data amount of the probe data obtained by using the encoding method of the present application, in which most of the frames are encoded, falls within the interval of 8 KB to 100 KB, and the data amount of the probe data in which a small number of frames are encoded is about 2,000 KB. It can be seen that the bit rate can be significantly reduced by using the encoding method of the present application.

[0444] 15 is a block diagram of an apparatus 1500 according to an embodiment of the present application. The apparatus 1500 includes a processor 1501 and a communication interface 1502, and may optionally further include a memory 1503.

[0445] The components of device 1500 are coupled together by bus 1504. In addition to a data bus, bus 1504 may further include a power bus, a control bus, and a status signal bus. However, for clarity of description, the various types of buses in the figures are referred to as bus 1504 in the figures.

[0446] Optionally, the memory 1503 may be configured to store instructions in the above method embodiments. The processor 1501 may be configured to execute the instructions in the memory 1503 and control the communication interface to receive / transmit signals.

[0447] The apparatus 1500 may be an electronic device or a chip within an electronic device in the above method embodiments.

[0448] For example, processor 1501 may be configured to process data, control data access and storage, issue commands, and control other components to perform operations. Processor 1501 may be implemented as one or more processors, one or more controllers, and / or other structures that can be used to execute programs. Processor 1501 may further include at least one of a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic component. General-purpose processors may include microprocessors, and any conventional processor, controller, microcontroller, or state machine. Processor 1501 may alternatively be implemented as a combination of computing components, e.g., a DSP and a microprocessor.

[0449] The memory 1503 may include a computer-readable storage medium, such as a magnetic storage device (e.g., a hard disk, a floppy disk, and a magnetic stripe), an optical storage medium (e.g., a digital versatile disk (DVD)), a smart card, a flash device, a random access memory (RAM), a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), a register, and any combination thereof. The memory 1503 may be coupled to the processor 1501 such that the processor 1501 can read information from and write information to the memory 1503. Specifically, the memory 1503 may be incorporated into the processor 1501, or the memory 1503 and the processor 1501 may be separate.

[0450] The communication interface 1502 may include circuitry and / or programming for implementing bidirectional communication between the terminal 1500 and one or more wireless networks (e.g., routers, switches, and access points). The communication interface 1502 includes at least one receiving circuit and / or at least one transmitting circuit. In an embodiment, the communication interface 1502 may be implemented partially or completely by a wireless modem.

[0451] All relevant contents of the steps in the above method embodiments can be cited in the functional descriptions of the corresponding functional modules, and the details will not be described again here.

[0452] The embodiments further provide a computer-readable storage medium having stored thereon computer instructions that, when executed on an electronic device, can cause the electronic device to perform the associated method steps described above to implement the encoding method of the above embodiments.

[0453] The embodiments further provide a computer program product, which, when run on a computer, enables the computer to perform the relevant steps above to implement the encoding method of the above embodiments.

[0454] Furthermore, an embodiment of the present application further provides an apparatus. The apparatus may specifically be a chip, a component, or a module. The apparatus may include a processor and a memory connected thereto. The memory is configured to store computer-executable instructions. When the apparatus operates, the processor may execute the computer-executable instructions in the memory to enable the chip to perform the encoding method in the above method embodiment.

[0455] The electronic device, computer-readable storage medium, computer program product, or chip provided in the embodiments is configured to perform the corresponding method provided above. Therefore, for the advantageous effects that can be achieved, please refer to the advantageous effects of the corresponding method provided above. Details will not be described here.

[0456] Based on the above description of the implementation, those skilled in the art can understand that the above division into functional modules is used as an example for explanation, for convenience and concise description. In actual applications, the above functions can be assigned to different functional modules and implemented according to requirements. In other words, the internal structure of the device is divided into different functional modules to implement all or part of the above functions.

[0457] It should be understood that in some embodiments provided herein, the disclosed devices and methods may be implemented in other manners. For example, the described device embodiments are merely examples. For example, the division into modules or units is merely a logical division of function, and other divisions may be used in actual implementation. For example, multiple units or components may be combined or integrated into other devices, or some features may be omitted or not implemented. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be implemented through some interface. Indirect couplings or communication connections between devices or units may be implemented electronically, mechanically, or in other forms.

[0458] The units described as separate parts may or may not be physically separated, and the parts shown as units may be one or more physical units, located in one place, or distributed in different places. Some or all of the units may be selected based on actual requirements to achieve the objectives of the solutions of the embodiments.

[0459] Furthermore, the functional units of the embodiments of the present application may be integrated into one processing unit, or each unit may exist physically independently, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0460] Any content of the embodiments of the present application and any content of the same embodiment can be freely combined. Any combination of the above content falls within the scope of the present application.

[0461] When an integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, the integrated unit may be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application may essentially, or the portion contributing to the prior art, or all or part of the technical solutions may be implemented in the form of a software product. The software product is stored in a storage medium and includes several instructions for instructing a device (which may be a single-chip microcomputer, a chip, etc.) or a processor to perform all or part of the steps of the method described in the embodiments of the present application. The storage medium includes any medium capable of storing program code, such as a USB flash drive, a removable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, etc.

[0462] The above describes the embodiments of the present application with reference to the accompanying drawings. However, the present application is not limited to the above specific implementations. The above specific implementations are merely examples and are not limiting. Inspired by the present application, those skilled in the art may make further modifications without departing from the purpose of the present application and the protection scope of the claims, and all modifications should fall within the protection scope of the present application.

[0463] The method or algorithm steps described in connection with the contents disclosed in the embodiments of the present application may be implemented by hardware or by a processor executing software instructions. The software instructions may include corresponding software modules. The software modules may be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, a hard disk, a removable hard disk, a compact disk read-only memory (CD-ROM), or any other form of storage medium known in the art. For example, the storage medium may be coupled to the processor, thereby enabling the processor to read information from and write information to the storage medium. Indeed, the storage medium may be a component of the processor. The processor and the storage medium may be located in an ASIC.

[0464] Those skilled in the art will recognize that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented by hardware, software, firmware, or any combination thereof. If the functions are implemented by software, they may be stored on a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer-readable storage media and communication media. Communication media include any medium that enables a computer program to be transmitted from one place to another. Storage media may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0465] The above describes the embodiments of the present application with reference to the accompanying drawings. However, the present application is not limited to the above specific implementations. The above specific implementations are merely examples and are not limiting. Inspired by the present application, those skilled in the art may make further modifications without departing from the purpose of the present application and the protection scope of the claims, and all modifications should fall within the protection scope of the present application.

[0466] This application claims priority from Chinese Patent Application No. 202210255740.4, entitled "ENCODING METHOD AND ELECTRONIC DEVICE," filed with the China Patent and Intellectual Property Office on March 15, 2022, which is incorporated herein by reference.

Claims

1. 1. An encoding method applied to a first device, comprising: acquiring probe data corresponding to one or more probes within a three-dimensional scene, the probe data being used by a second device to determine shading effects of objects within the three-dimensional scene in a rendering process; performing a data format transformation on the probe data to obtain intermediate data, the data format transformation comprising a domain transformation; and encoding said intermediate data to obtain a corresponding bitstream; and The above-mentioned performing data format conversion on the probe data to obtain intermediate data includes: performing a first processing on the probe data to obtain transformed data; performing a second operation on the transformed data to obtain the intermediate data; and and If the first processing is a domain transformation, the second processing comprises quantization and / or first modality rearrangement; If the second processing is the domain transformation, the first processing comprises the quantization and / or the first modality rearrangement. method.

2. The above-mentioned performing data format conversion on the probe data to obtain intermediate data includes: The method further includes performing a third processing on the probe data before performing the first processing on the probe data. The third processing comprises at least one of the following: the domain transformation, the quantization, and the first modality rearrangement. The method of claim 1.

3. After the second processing is performed on the transformed data and before the intermediate data is obtained, the method further comprises: performing a second style rearrangement on the data obtained by the second process to obtain the intermediate data; the second manner rearrangement is padding the data obtained by the second processing into a YUV plane; The method of claim 1.

4. When the probe data is illuminance data, and when the illuminance data has multiple channels, the first style rearrangement includes the following steps: Discarding data of some channels, converting the precision of the illuminance data to a preset precision, or performing a dimensional transformation on the illuminance data. having at least one of: The method of claim 1.

5. When the probe data is visibility data, and the visibility data has multiple channels, the first modality rearrangement includes: performing channel splitting, converting the precision of the visibility data to a preset precision, or performing a dimensional transformation on the visibility data; having at least one of: The method of claim 1.

6. The domain transformation comprises at least one of the following: a transformation from a non-normalized domain to a normalized domain, a transformation from a linear domain to a non-linear domain, a transformation from an RGB domain to a YUV domain, a transformation from an RGB domain to an XYZ domain, a transformation from an RGB domain to a Lab domain, and a transformation from a picture domain to a transform domain. The method of claim 1.

7. The probe data is represented by a two-dimensional picture, spherical harmonic basis coefficients, or spherical wavelet basis coefficients; The method of claim 1.

8. The method comprises: encoding attribute data of the one or more probes to obtain the bitstream; The attribute data includes first attribute data for the data format conversion and / or second attribute data for use in the rendering process. The method of claim 1.

9. When the probe data includes illuminance data and visibility data, the bitstream includes bitstream structure information, and the bitstream structure information includes a position of intermediate data corresponding to the illuminance data and / or a position of intermediate data corresponding to the visibility data. The method of claim 1.

10. If the probe data comprises illuminance data and visibility data, the method further comprises: determining a first target bit rate corresponding to the illuminance data and a second target bit rate corresponding to the visibility data based on a data amount of the illuminance data, a data amount of the visibility data, and a preset bit rate; said encoding said intermediate data to obtain a corresponding bitstream comprising: encoding the intermediate data corresponding to the illuminance data based on the first target bit rate, and encoding the intermediate data corresponding to the visibility data based on the second target bit rate to obtain the bit rate. The method of claim 1.

11. If the probe data comprises illuminance data and visibility data, the method further comprises: determining an encoding scheme corresponding to the illuminance data and an encoding scheme corresponding to the visibility data based on a data amount of the illuminance data, a data amount of the visibility data, and channel feedback information, wherein the encoding scheme comprises intra-frame encoding or inter-frame encoding; said encoding said intermediate data to obtain a corresponding bitstream comprising: encoding the intermediate data corresponding to the illuminance data by using the encoding scheme corresponding to the illuminance data, and encoding the intermediate data corresponding to the visibility data by using the encoding scheme corresponding to the visibility data to obtain the bitstream. The method of claim 1.

12. a first device, a data acquisition module configured to acquire probe data corresponding to one or more probes in a three-dimensional scene, the probe data being used by a second device to determine shading effects of objects in the three-dimensional scene in a rendering process; and a data format conversion module configured to perform a data format conversion on the probe data to obtain intermediate data, the data format conversion comprising a domain conversion; and an encoding module configured to encode the intermediate data to obtain a corresponding bitstream; and The data format conversion module performing a first process on the probe data to obtain transformed data; performing a second process on the transformed data to obtain the intermediate data; It is configured as If the first processing is a domain transformation, the second processing comprises quantization and / or first modality rearrangement; If the second processing is the domain transformation, the first processing comprises the quantization and / or the first modality rearrangement. First device.

13. A first device configured to perform the encoding method according to any one of claims 1 to 11.

14. 1. An electronic device having a memory and at least one processor, the memory is coupled to the at least one processor; The memory stores program instructions that, when executed by the at least one processor, enable the electronic device to perform the encoding method of any one of claims 1 to 11. Electronic devices.

15. A computer program which, when executed by an electronic device, causes the electronic device to perform the encoding method of any one of claims 1 to 11.

Images