Texture-image encoding method, texture-image decoding method, and electronic device

Abstract

Provided in the embodiments of the present application are a texture-image encoding method, a texture-image decoding method, and an electronic device. The encoding method comprises: first, performing fitting processing on pixel values of a plurality of texture pixels in a texture image, so as to obtain a geometric surface; then, acquiring the coordinates of a plurality of projection points in the geometric surface, wherein the plurality of projection points correspond to the pixel values of the plurality of texture pixels on a one-to-one basis, and the coordinates of the projection points comprise two or more dimensions; next, acquiring the values of three or more endpoints used for representing the geometric surface; and subsequently, encoding the values of the three or more endpoints, and encoding the coordinates of the plurality of projection points, so as to obtain an encoded bitstream. The method can reduce fitting errors of texture images, thereby improving the quality of texture-image reconstruction and improving the quality of rendered images.

Description

Texture image encoding and decoding method and electronic device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on February 7, 2024, with application number 202410174008.3 and application name “Texture Image Coding and Decoding Method and Electronic Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of coding and decoding, and in particular to a texture image coding and decoding method and electronic equipment. Background Art

[0003] Texture mapping, also known as texture mapping, is the process of mapping texture pixels (i.e., pixel values ​​of a texture image) in texture space to the surface of a 3D model. It can enhance the realism of the surface of a three-dimensional object without increasing geometric complexity and is one of the important technologies in graphics rendering.

[0004] 3D applications consume a large amount of memory when running, with over half of this memory being used by texture images. Texture compression can reduce this memory usage. Furthermore, during rendering, large numbers of texture images must be transferred from memory to the graphics processing unit (GPU), creating a bandwidth bottleneck. Texture compression can alleviate bus bandwidth congestion and reduce power consumption.

[0005] Currently, texture compression technology based on straight line fitting is used to achieve texture image compression; however, this will produce large errors, resulting in poor quality of the decoded texture image, which in turn leads to poor quality of the rendered image. Summary of the Invention

[0006] In view of this, the present application provides a texture image encoding and decoding method and electronic device, which can reduce the fitting error of the texture image, thereby improving the reconstruction quality of the texture image and improving the quality of the rendered image.

[0007] In a first aspect, the present application provides a method for encoding a texture image, the method comprising: first, performing fitting processing on the pixel values ​​of multiple texture pixels in the texture image to obtain a geometric surface; then, obtaining the coordinates of multiple projection points in the geometric surface; wherein the multiple projection points correspond one-to-one to the pixel values ​​of the multiple texture pixels, and the coordinates of the projection points include two or more dimensions; then, obtaining values ​​for representing three or more endpoints of the geometric surface; thereafter, encoding the values ​​of the three or more endpoints, and encoding the coordinates of the multiple projection points to obtain an encoded bitstream.

[0008] Since the surface properties of spatial objects (such as color, normal, etc.) are mostly continuous; therefore, the texture image has the characteristics of block-like planar distribution, and these characteristics can constitute a continuous surface in space. Furthermore, compared with the prior art that fits the pixel values ​​of texture pixels in the texture image to a straight line, the present application fits the pixel values ​​of texture pixels in the texture image to a geometric surface, which can reduce the fitting error. In other words, compared with the prior art, the present application can reduce the error between the pixel value of the texture pixel and the coordinates of the corresponding projection point; in this way, the error between the reconstructed value of the texture pixel reconstructed by the present application based on the encoded data (that is, the coordinates of the encoded projection point) and the pixel value of the texture pixel (that is, the original value) is smaller (that is, the encoding error is smaller). When the bit rate of the encoded texture image is fixed (that is, at the same bit rate), the reconstructed value of the texture pixel reconstructed by the present application is more accurate compared with the prior art; thus, the present application can improve the reconstruction quality of the texture image and the quality of the rendered image.

[0009] It should be noted that the pixel value of the encoded texture image is the coordinate of the encoded projection point; the values ​​used to represent three or more endpoints of the geometric surface can be regarded as auxiliary information used to reconstruct the reconstruction value of the texture pixel in the texture image; the encoding value representing three or more endpoints of the geometric surface is the encoding auxiliary information.

[0010] For example, the pixel values ​​of texture pixels in the texture image may be used to represent color information (such as RGB, YUV, etc.) of vertices on the three-dimensional model.

[0011] Exemplarily, the pixel value of the texture pixel in the texture image may also be material information of the three-dimensional model, such as roughness, normal information, ambient light information, etc., which is not limited in this application.

[0012] For example, a texture image may include at least two channels, and a pixel value of a texture pixel in the texture image may include at least two components. It should be noted that the number of components included in the pixel value of the texture pixel is the same as the number of channels included in the texture image; and the multiple components included in the pixel value of the texture pixel correspond one-to-one to the multiple channels of the texture image.

[0013] For example, a texture image includes three channels (R channel, G channel, and B channel), and correspondingly, a pixel value of a texture pixel includes three components: R component, G component, and B component; in this case, the texture image can be called an RGB image.

[0014] For another example, a texture image includes two channels (a roughness channel and an ambient light channel), and correspondingly, a pixel value of a texture pixel includes two components: a roughness component and an ambient light component.

[0015] For another example, a texture image includes four channels (R channel, G channel, B channel and A (transparency) channel), and correspondingly, the pixel value of a texture pixel may include four components: R component, G component, B component and A component.

[0016] For example, the pixel value of one texture pixel may correspond to one projection point. For example, if the texture image includes 64*64 texture pixels, then it may correspond to 64*64 fitting points.

[0017] Exemplarily, a fitting point can be determined based on the pixel value of a texture pixel. Specifically, a component of the pixel value of the texture pixel can be used as a dimension of the coordinates of the fitting point. Here, a dimension of the coordinates of the fitting point can be understood as a component of the coordinates of the fitting point. For example, the pixel value of the texture pixel P1 is (R1, G1, B1), that is, the pixel value of the texture pixel P1 includes three components, then the coordinates of the corresponding fitting point are (R1, G1, B1); that is, the coordinates of the corresponding fitting point also include three components (or three dimensions); the three components of the pixel value of the texture pixel P1 correspond one-to-one to the three components of the coordinates of the corresponding fitting point. Exemplarily, a fitting point can be projected onto a geometric surface to obtain a projection point.

[0018] Exemplarily, among the three or more endpoints used to represent the geometric surface, any endpoint may include multiple components; the number of components contained in each endpoint is the same as the number of channels contained in the texture image; and the multiple components contained in each endpoint correspond one-to-one to the multiple channels contained in the texture image.

[0019] For example, the geometric surface may be a plane, a curved surface, or a geometric body.

[0020] For example, when the geometric surface is a plane or a curved surface, three endpoints representing the plane or the curved surface may be obtained.

[0021] Exemplarily, when the geometric surface is a geometric body, four endpoints for representing the geometric body may be obtained.

[0022] Exemplarily, a data optimization method can be used to fit the pixel values ​​of multiple texture pixels in a texture image to obtain an objective function, which is used to represent a geometric surface; that is, by fitting the pixel values ​​of multiple texture pixels in a texture image, a geometric surface can be obtained.

[0023] It should be noted that the present application does not limit the data optimization method used in the fitting process, for example, it may be principal component analysis (PCA), least squares method, gradient descent method, etc.

[0024] For example, the dimensions of the projection point coordinates can be smaller than the number of channels in the texture image. For example, if the texture image contains three channels, the projection point coordinates include two dimensions; or, for another example, if the texture image contains four channels, the projection point coordinates include three dimensions.

[0025] Exemplarily, when the geometric surface is a plane or a curved surface, the coordinates of the projection point may include two dimensions, for example, the coordinates of the projection point are (u, v).

[0026] Exemplarily, when the geometric surface is a geometric body, the coordinates of the projection point may include more than two dimensions.

[0027] Exemplarily, the coordinates of multiple projection points can also be quantized, as well as the values ​​of three or more endpoints used to represent a geometric surface. Then, the quantized values ​​of the three or more endpoints used to represent the geometric surface and the quantized coordinate values ​​of multiple projection points are encoded.

[0028] In one possible approach, the values ​​of three or more endpoints used to represent a geometric surface may be preprocessed (or called pre-processed) to obtain pre-processed values ​​of the three or more endpoints; thereafter, the pre-processed values ​​of the three or more endpoints may be quantized and encoded.

[0029] In one possible approach, assuming that the three or more endpoints used to represent a geometric surface include a first portion of endpoints and a second portion of endpoints, the values ​​of the first portion of endpoints may be preprocessed and then quantized and encoded; and the values ​​of the second portion of endpoints may be directly quantized and encoded.

[0030] In one possible approach, when the geometric surface is a plane, three endpoints that can form a vertical coordinate system and enable the coordinates of all projection points to be within the range of [0, 1] can be taken from the plane as the three endpoints used to represent the plane.

[0031] In one possible approach, when the geometric surface is a plane, three endpoints that can form a non-vertical coordinate system and enable the coordinates of all projection points to be within the range of [0,1] can be taken from the plane as the three endpoints used to represent the plane.

[0032] It should be understood that when selecting endpoints in a plane, three endpoints can also be taken from the plane that can form a vertical coordinate system (or non-vertical coordinate system) and can make the coordinates of all projection points within other ranges that are easy to encode, as the three endpoints used to represent the plane; this application does not impose any restrictions on this.

[0033] For example, when the geometric surface is an elliptical paraboloid, the three endpoints selected to represent the elliptical paraboloid may be: the vertex of the elliptical paraboloid, the direction of the axis of symmetry, and the coefficient of the elliptical equation. It should be understood that the three endpoints used to represent different curved surfaces may be different.

[0034] For example, when the geometric surface is an ellipsoid, the four endpoints selected to represent the ellipsoid may be: the center of the ellipsoid, the directions of the two symmetry axes, and the coefficients of the ellipse equation. It should be understood that the three or more endpoints used to represent different geometric bodies may be different.

[0035] Exemplarily, the encoded code stream may also be referred to as a code stream, a bit stream or a bit stream.

[0036] For example, the texture image encoding method of the present application can be applied to 3D applications. For example, 3D applications may include, but are not limited to: cloud gaming, virtual reality (VR) / augmented reality (AR) applications (such as AR / VR conferences), digital human live broadcasts, digital museums, virtual concerts, etc., and this application does not limit this.

[0037] According to the first aspect, three or more endpoints include: a first endpoint and a second endpoint; encoding the value of the first endpoint and the value of the second endpoint includes: determining the residual between the value of the second endpoint and the value of the first endpoint as the residual value of the second endpoint; wherein the value of the first endpoint is used as the predicted value of the second endpoint; encoding the residual value of the second endpoint and the value of the first endpoint.

[0038] In this way, when the residual value of the second endpoint is small, the bits required to encode the residual value of the second endpoint are fewer than the bits required to encode the value of the second endpoint. When the bit rate of the encoded texture image is fixed, the bits used to encode the value of the first endpoint can be increased. This can improve the accuracy of the reconstructed value of the first endpoint obtained by the decoder, as well as the accuracy of the reconstructed value of the second endpoint determined based on the reconstructed value of the first endpoint and the reconstructed value of the residual value of the second endpoint. This can improve the quality of the texture image reconstruction and the quality of the rendered image. When the bit rate of the encoded texture image is not fixed, the bit rate can be reduced.

[0039] According to the first aspect, or any implementation method of the first aspect above, the three or more endpoints also include: other endpoints; encoding the values ​​of other endpoints, including: determining the residual between the value of the other endpoint and the value of the first endpoint as the residual value of the other endpoint; wherein the value of the first endpoint is used as the predicted value of the other endpoint; encoding the residual value of the other endpoint.

[0040] Similarly, when the residual values ​​of other endpoints are small, fewer bits are required to encode the residual values ​​of the other endpoints compared to the bits required to encode the values ​​of the other endpoints. When the bitrate of the encoded texture image is fixed, the number of bits used to encode the value of the first endpoint can be increased. This improves the accuracy of the reconstructed value of the first endpoint obtained by the decoder, as well as the accuracy of the reconstructed values ​​of the other endpoints determined based on the reconstructed value of the first endpoint and the reconstructed values ​​of the residual values ​​of the other endpoints. When the bitrate of the encoded texture image is not fixed, the bitrate can be reduced.

[0041] Exemplarily, when there are multiple other endpoints, the residual value of each other endpoint can be encoded.

[0042] In one possible approach, the values ​​of other endpoints can also be encoded directly.

[0043] In one possible approach, when there are two other endpoints (including a third endpoint and a fourth endpoint), the residual between the value of the third endpoint and the value of the fourth endpoint (ie, the residual value of the fourth endpoint) can also be encoded, as well as the value of the third endpoint.

[0044] Similarly, when there are multiple other endpoints, the residual between the values ​​of any two endpoints and the value of any one of the two endpoints can be encoded; this application does not impose any restrictions on this.

[0045] According to the first aspect, or any implementation of the first aspect above, encoding the residual value of the second endpoint and the value of the first endpoint includes: encoding the residual value of the second endpoint and the value of the first endpoint when the residual value of the second endpoint is within a preset range.

[0046] When the residual between the value of the first endpoint and the value of the second endpoint is within a preset range, the number of bits required to encode the residual value of the second endpoint can be determined to be smaller than the number of bits required to directly encode the value of the second endpoint. This increases the number of bits used to encode the endpoint value of the first endpoint. Correspondingly, the accuracy of the endpoint reconstructed value of the first endpoint obtained by the decoding end is relatively high, and the accuracy of the endpoint reconstructed value of the second endpoint reconstructed based on the endpoint reconstructed value of the first endpoint and the residual is also relatively high.

[0047] For example, when the residual difference between the values ​​of the first endpoint and the second endpoint is not within a preset range, the values ​​of the first endpoint and the second endpoint can be directly encoded. This is because when the residual difference between the values ​​of the first endpoint and the second endpoint is not within the preset range, that is, the difference between the values ​​of the first endpoint and the second endpoint is large; in this case, the number of bits required to encode the residual difference between the values ​​of the first endpoint and the second endpoint is equal to the number of bits required to encode the value of the second endpoint. This way, the number of bits used to encode the first endpoint does not increase, and the accuracy of the reconstructed value of the first endpoint obtained by the decoder does not improve. Because the reconstructed value of the first endpoint has errors, and the reconstructed value of the residual value of the second endpoint also has errors, the reconstructed value of the second endpoint reconstructed based on the reconstructed value of the first endpoint and the residual value of the second endpoint will have a larger error than the error of the reconstructed value of the second endpoint directly decoded from the encoded bitstream, thereby reducing the accuracy of the reconstructed value of the second endpoint. Therefore, to ensure the accuracy of the reconstructed value of the second endpoint, the values ​​of the first endpoint and the second endpoint can be directly encoded.

[0048] Exemplarily, when the residual values ​​of the other endpoints are within a preset range, the residual values ​​of the other endpoints are encoded.

[0049] Exemplarily, when there are two other endpoints (including a third endpoint and a fourth endpoint), when the residual between the value of the third endpoint and the value of the fourth endpoint (i.e., the residual value of the fourth endpoint) is within a preset range, the residual between the value of the third endpoint and the value of the fourth endpoint (i.e., the residual value of the fourth endpoint) is encoded, as well as the value of the third endpoint.

[0050] It should be understood that when the residual between the values ​​of any two endpoints used to represent a geometric surface is obtained, it is possible to first determine whether the residual is within a preset range. When it is within the preset range, the residual between the values ​​of the two endpoints and the value of any one of the two endpoints are encoded.

[0051] According to the first aspect, or any implementation method of the first aspect above, the pixel values ​​of multiple texture pixels in the texture image are fitted to obtain a geometric surface, including: fitting the pixel values ​​of multiple texture pixels to obtain multiple candidate geometric surfaces; selecting a geometric surface from the multiple candidate geometric surfaces based on the types of the multiple candidate geometric surfaces; wherein the type includes any one of the following: a plane, a curved surface, or a geometric body.

[0052] In one possible approach, multiple types of candidate geometric surfaces may be used to fit texture pixels.

[0053] In one possible approach, a type of candidate geometric surface may be used to fit the texture pixels.

[0054] According to the first aspect, or any implementation method of the first aspect above, a geometric surface is selected from multiple candidate geometric surfaces based on their types, including: when the types of multiple candidate geometric surfaces are the same, one of the N candidate geometric surfaces with the smallest corresponding fitting error is determined as the geometric surface, where N is a positive integer; wherein the fitting error corresponding to the current candidate geometric surface among the multiple candidate geometric surfaces is determined based on the sum of the distances between the pixel values ​​of multiple texture pixels and the current candidate geometric surface.

[0055] It should be noted that the distance between the pixel value of a texture pixel and the current candidate geometric surface may refer to the distance between a fitting point determined based on the pixel value of a texture pixel (also referred to as a fitting point corresponding to the pixel value of a texture pixel) and the current candidate geometric surface.

[0056] Among them, for the same type of geometric surface, the smaller the fitting error of the geometric surface, the smaller the error between the reconstructed value of the texture pixel reconstructed according to the encoded data (that is, the coordinates of the encoded projection point) and the pixel value (that is, the encoding error is small); in this way, the reconstructed value of the reconstructed texture pixel is more accurate.

[0057] Optionally, N is 1, that is, the candidate geometric surface with the smallest fitting error is determined as the geometric surface. In this way, the reconstructed values ​​of the reconstructed texture pixels can be made most accurate.

[0058] According to the first aspect, or any implementation method of the first aspect above, a geometric surface is selected from multiple candidate geometric surfaces based on the types of multiple candidate geometric surfaces, including: when the types of at least two candidate geometric surfaces among the multiple candidate geometric surfaces are different, the candidate geometric surface with the smallest corresponding coding error is determined as the geometric surface; wherein the coding error corresponding to the current candidate geometric surface among the multiple candidate geometric surfaces is the error of the encoding used to represent the values ​​of three or more candidate endpoints of the current candidate geometric surface and the coordinates of multiple candidate projection points in the current candidate geometric surface, and the multiple candidate projection points correspond one-to-one to the pixel values ​​of multiple texture pixels.

[0059] For different types of geometric surfaces, the smaller the coding error is, the more accurate the reconstructed value of the reconstructed texture pixel is.

[0060] According to the first aspect, or any implementation of the first aspect above, encoding the coordinates of multiple projection points includes: encoding the coordinates of the multiple projection points using a combination of encoding algorithms that minimizes encoding errors.

[0061] According to the first aspect, or any implementation method of the first aspect above, a coding algorithm combination with minimal coding error is used to encode the coordinates of multiple projection points, including: when the coding algorithm combination with minimal coding error includes two or more coding algorithms, for a first texture pixel among multiple texture pixels, according to a coding algorithm in the coding algorithm combination with minimal coding error, encode a dimension of the coordinates of the first projection point; wherein the first projection point corresponds to the pixel value of the first texture pixel.

[0062] For example, when the coding algorithm combination with the minimum coding error includes multiple coding algorithms (the number of coding algorithms is the same as the dimensions included in the coordinates of the projection points), one coding algorithm in the coding algorithm combination with the minimum coding error is used to encode one dimension of the coordinates of each projection point. For example, if the coordinates of the projection point A1 of the fitting point A onto the geometric surface are (u1, v1), u1 can be encoded according to one coding algorithm included in the coding algorithm combination with the minimum coding error, and v1 can be encoded according to another coding algorithm included in the coding algorithm combination with the minimum coding error.

[0063] For example, when the coding algorithm combination with the minimum coding error includes multiple coding algorithms (the number of coding algorithms is greater than 1 and less than the dimensions included in the coordinates of the projection point), one coding algorithm in the coding algorithm combination with the minimum coding error is used to encode at least one dimension of the coordinates of each projection point. For example, if the coordinates of the projection point A1 of the fitting point A to the geometric surface are (u1, v1, r1), u1 and v1 can be encoded according to one coding algorithm included in the coding algorithm combination with the minimum coding error; and r1 can be encoded according to another coding algorithm included in the coding algorithm combination with the minimum coding error.

[0064] Illustratively, the coding algorithm combination with the smallest coding error includes two or more coding algorithms, which may be the same coding algorithm or different coding algorithms, and this application does not impose any limitation on this.

[0065] According to the first aspect, or any implementation method of the first aspect above, a coding algorithm combination with minimal coding error is used to encode the coordinates of multiple projection points, including: when the coding algorithm combination with minimal coding error includes one coding algorithm, for a second texture pixel among multiple texture pixels, according to one coding algorithm in the coding algorithm combination with minimal coding error, all dimensions of the coordinates of the second projection point are encoded; wherein the second projection point corresponds to the pixel value of the second texture pixel.

[0066] For example, when the coding algorithm combination with the minimum coding error includes a coding algorithm, the coding algorithm can be used to encode the entirety of the coordinates of two or more dimensions of each projection point. For example, if the coordinates of the projection point A1 of the fitting point A to the geometric surface are (u1, v1), the entirety of u1 and v1 can be encoded according to a coding algorithm included in the coding algorithm combination with the minimum coding error. For example, after concatenating u1 and v1, the concatenated data is encoded according to a coding algorithm included in the coding algorithm combination with the minimum coding error.

[0067] For example, when the coding algorithm combination with the minimum coding error includes a coding algorithm, the coding algorithm can be used to independently encode the coordinates of two or more dimensions of each projection point. For example, if the coordinates of the projection point A1 of the fitting point A onto the geometric surface are (u1, v1), the coding algorithm combination with the minimum coding error includes a coding algorithm to independently encode u1 and v1.

[0068] According to the first aspect, or any implementation of the first aspect above, the texture image includes multiple texture blocks, and the multiple texture pixels belong to any one of the multiple texture blocks. In other words, fitting processing can be performed on a texture block basis to obtain a geometric surface. Thus, encoding of the texture image is achieved by repeatedly executing the texture image encoding method of this application. In this case, for a texture block, only one geometric surface is determined; for the entire texture image, multiple geometric surfaces are determined.

[0069] According to the first aspect, or any implementation of the first aspect above, the texture image includes multiple texture blocks, each texture block includes multiple texture regions, and multiple texture pixels belong to any one texture region. In other words, fitting processing can be performed on a region-by-region basis to obtain a geometric surface. Thus, encoding of the texture image is achieved by repeatedly executing the texture image encoding method of this application. In this case, for a texture region, one geometric surface is determined; for the entire texture image, multiple geometric surfaces are determined.

[0070] It should be noted that the fitting process can also be performed on a texture image basis to obtain a geometric surface.

[0071] In a second aspect, the present application provides an encoded code stream, which is generated according to the first aspect and any implementation manner of the first aspect.

[0072] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0073] In a third aspect, the present application provides an encoded bitstream, which includes the encoded values ​​of three or more endpoints and the encoded coordinates of multiple projection points; wherein the three or more endpoints are used to represent a geometric surface, and the geometric surface is obtained by fitting the pixel values ​​of multiple texture pixels in a texture image, and the geometric surface includes multiple projection points, and the multiple projection points correspond one-to-one to the pixel values ​​of multiple texture pixels, and the coordinates of the projection points include two or more dimensions.

[0074] According to the third aspect, the three or more endpoints include a first endpoint and a second endpoint; the encoded code stream also includes a first identifier, the first identifier indicates whether to encode the residual value of the second endpoint, and the residual value of the second endpoint is the residual between the value of the first endpoint and the value of the second endpoint.

[0075] Exemplarily, the first identifier may also indicate an encoding algorithm for representing the values ​​of three or more endpoints of a geometric surface.

[0076] Exemplarily, the first identifier may also indicate a storage format of encoded data representing the values ​​of three or more endpoints of a geometric surface.

[0077] Exemplarily, the first identifier may also be used to indicate a reference endpoint of the current endpoint.

[0078] According to the third aspect, or any implementation of the third aspect above, the encoded code stream further includes a second identifier, and the second identifier indicates the encoding algorithm corresponding to the coordinates of each projection point in the multiple projection points.

[0079] Exemplarily, the second identifier may further indicate whether residual processing is performed on the coordinates of the multiple projection points.

[0080] Exemplarily, the second identifier may also indicate a storage format of the encoded data of the coordinates of the plurality of projection points.

[0081] According to the third aspect, or any implementation of the third aspect above, the second identifier indicates an encoding algorithm corresponding to each dimension of the coordinates of each projection point in the multiple projection points.

[0082] According to the third aspect, or any implementation of the third aspect above, the encoded code stream further includes a third identifier, and the third identifier indicates the type of the geometric surface.

[0083] It should be understood that the present application is not limited to the encoded data used to represent the values ​​of three or more endpoints of a geometric surface, the encoded data of the coordinates of multiple projection points, and the order of the first identifier, the second identifier, and the third identifier in the encoded code stream.

[0084] It should be understood that the codec can pre-agreed on the encoding algorithm used to represent the values ​​of three or more endpoints of a geometric surface, whether to perform residual processing on the values ​​of three or more endpoints of a geometric surface, the storage format of the encoded data used to represent the values ​​of three or more endpoints of a geometric surface in the encoded bitstream, and the reference endpoint of the current endpoint. In this way, there is no need to carry the first identifier in the encoded bitstream.

[0085] It should be understood that the codec can pre-agreed on the encoding algorithm for the coordinates of the multiple projection points, whether the coordinates of the multiple projection points are subjected to residual processing, and the storage format of the encoded data of the coordinates of the multiple projection points. In this way, there is no need to carry the second identifier in the encoded bitstream.

[0086] It should be understood that the codec can pre-agreed on the type of geometric surface; in this way, there is no need to carry the third identifier in the encoded bitstream.

[0087] That is, the first, second, and third identifiers are optional in the encoded bitstream. When the encoded bitstream does not carry at least one of the first, second, and third identifiers, the number of bits used to encode the coordinates of multiple projection points or the number of bits used to encode the values ​​of three or more endpoints representing a geometric surface can be increased; thereby, the encoding accuracy of the coordinates of the multiple projection points or the encoding accuracy of the values ​​of three or more endpoints representing a geometric surface can be increased.

[0088] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0089] In a fourth aspect, the present application provides a method for decoding a texture image, which comprises: first, obtaining an encoded code stream; then, decoding the encoded code stream to obtain reconstruction values ​​of three or more endpoints and coordinate reconstruction values ​​of multiple projection points; wherein the three or more endpoints are used to represent a geometric surface, the geometric surface includes multiple projection points, the multiple projection points correspond one-to-one to pixel values ​​of multiple texture pixels in the texture image, and the coordinate reconstruction values ​​include two or more dimensions; then, based on the reconstruction values ​​of the three or more endpoints and the coordinate reconstruction values ​​of the multiple projection points, the reconstruction values ​​of multiple texture pixels in the texture image are generated.

[0090] In one possible approach, during rendering, the terminal device or server may read the encoded code stream from the memory.

[0091] In one possible approach, during rendering, the terminal device may receive the encoded code stream sent by the server and store it in a memory, and then read the encoded code stream from the memory.

[0092] Exemplarily, the coordinate quantization values ​​of the plurality of projection points may be decoded from the encoded bitstream; then, the coordinate quantization values ​​of the plurality of projection points may be dequantized to obtain the coordinate reconstruction values ​​of the plurality of projection points.

[0093] It should be noted that the coordinate reconstruction value of each projection point obtained by decoding may be the same as or different from the coordinate of each projection point used for encoding, and this application does not impose any restrictions on this.

[0094] In one possible approach, quantized values ​​of three or more endpoints representing the geometric surface can be decoded from the encoded bitstream. Subsequently, the quantized values ​​of the three or more endpoints representing the geometric surface can be dequantized to obtain reconstructed values ​​of the three or more endpoints representing the geometric surface.

[0095] In one possible approach, quantized values ​​of three or more endpoints representing a geometric surface can be decoded from an encoded bitstream. Subsequently, the quantized values ​​of the three or more endpoints representing the geometric surface can be dequantized to obtain dequantized values ​​of the three or more endpoints representing the geometric surface. Thereafter, post-processing can be performed based on the dequantized values ​​of the three or more endpoints representing the geometric surface to obtain reconstructed values ​​of the three or more endpoints representing the geometric surface.

[0096] In one possible approach, the quantized values ​​of the first portion of endpoints and the quantized values ​​of the second portion of endpoints can be decoded from the bitstream. The quantized values ​​of the first portion of endpoints can then be dequantized to obtain dequantized values ​​of the first portion of endpoints. The quantized values ​​of the second portion of endpoints can then be dequantized to obtain reconstructed values ​​of the second portion of endpoints. Finally, post-processing can be performed based on the reconstructed values ​​of the second portion of endpoints and the dequantized values ​​of the first portion of endpoints to obtain reconstructed values ​​of the first portion of endpoints.

[0097] It should be noted that the reconstructed value of each endpoint obtained by decoding may be the same as or different from the value of each endpoint used for encoding, and this application does not impose any restrictions on this.

[0098] According to a fourth aspect, the three or more endpoints include: a first endpoint and a second endpoint; decoding the encoded code stream to obtain a reconstructed value of the first endpoint and a reconstructed value of the second endpoint, including: parsing a first identifier from the encoded code stream; wherein the first identifier indicates whether to encode a residual value of the second endpoint, and the residual value of the second endpoint is the residual between the value of the second endpoint and the value of the first endpoint; when the value of the first identifier is a preset identifier value, decoding the encoded code stream to obtain a reconstructed value of the first endpoint and a reconstructed value of the residual value of the second endpoint; adding the reconstructed value of the first endpoint to the reconstructed value of the residual value of the second endpoint to obtain the reconstructed value of the second endpoint; wherein the reconstructed value of the first endpoint is used as a predicted value of the second endpoint.

[0099] According to the fourth aspect, or any implementation method of the above fourth aspect, the three or more endpoints also include: other endpoints; decoding the encoded code stream to obtain the reconstructed values ​​of the other endpoints, including: when the value of the first identifier is a preset identifier value, decoding the encoded code stream to obtain the reconstructed values ​​of the residual values ​​of the other endpoints; wherein the residual values ​​of the other endpoints are the residuals between the values ​​of the other endpoints and the value of the first endpoint; adding the reconstructed value of the first endpoint to the reconstructed values ​​of the residual values ​​of the other endpoints to obtain the reconstructed values ​​of the other endpoints; wherein the reconstructed value of the first endpoint is used as the predicted value of the other endpoints.

[0100] According to the fourth aspect, or any implementation of the fourth aspect above, decoding the encoded code stream to obtain coordinate reconstruction values ​​of multiple projection points includes: parsing a second identifier from the encoded code stream; wherein the second identifier indicates a decoding algorithm corresponding to the coordinates of each projection point in the multiple projection points; decoding the encoded code stream according to the decoding algorithm indicated by the second identifier to obtain coordinate reconstruction values ​​of the multiple projection points.

[0101] According to the fourth aspect, or any implementation of the fourth aspect above, the second identifier indicates multiple decoding algorithms, and the encoded code stream is decoded according to the decoding algorithm indicated by the second identifier to obtain coordinate reconstruction values ​​of multiple projection points, including: using a decoding algorithm indicated by the second identifier to decode the decoded code stream to obtain reconstruction values ​​of one or more dimensions of the coordinates of each projection point in the multiple projection points.

[0102] According to the fourth aspect, or any implementation method of the above fourth aspect, the second identifier indicates a decoding algorithm, and the code stream is decoded according to the decoding algorithm indicated by the second identifier to obtain coordinate reconstruction values ​​of multiple projection points, including: decoding the code stream according to a decoding algorithm indicated by the second identifier to obtain reconstruction values ​​of all dimensions of the coordinates of each projection point in the multiple projection points.

[0103] According to the fourth aspect, or any implementation of the fourth aspect above, the reconstruction values ​​of multiple texture pixels in the texture image are determined based on the reconstruction values ​​of three or more endpoints and the coordinate reconstruction values ​​of multiple projection points, including: interpolating the reconstruction values ​​of the three or more endpoints and the coordinate reconstruction values ​​of the multiple projection points to obtain the reconstruction values ​​of the multiple texture pixels.

[0104] According to the fourth aspect, or any implementation of the fourth aspect above, the method also includes: parsing a third identifier from the code stream; wherein the third identifier indicates the type of the geometric surface; interpolating the reconstruction values ​​of three or more endpoints and the coordinate reconstruction values ​​of multiple projection points to obtain reconstruction values ​​of multiple texture pixels, including: using the interpolation algorithm corresponding to the type of the geometric surface indicated by the third identifier, interpolating the reconstruction values ​​of three or more endpoints and the coordinate reconstruction values ​​of multiple projection points to obtain reconstruction values ​​of multiple texture pixels.

[0105] The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0106] In a fifth aspect, the present application provides an encoder comprising: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, which, when executed by the processor, enable the encoder to execute the method of the first aspect or any possible implementation of the first aspect.

[0107] The fifth aspect and any implementation of the fifth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0108] In a sixth aspect, the present application provides a decoder comprising: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, which, when executed by the processor, enable the decoder to execute the method in the fourth aspect or any possible implementation of the fourth aspect.

[0109] The sixth aspect and any implementation of the sixth aspect correspond to the fourth aspect and any implementation of the fourth aspect, respectively. The technical effects corresponding to the sixth aspect and any implementation of the sixth aspect can be referred to the technical effects corresponding to the fourth aspect and any implementation of the fourth aspect, and will not be repeated here.

[0110] In the seventh aspect, the present application provides a chip comprising one or more interface circuits and one or more processors; the interface circuit is used to receive signals from a memory of an electronic device and send signals to the processor, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the method in the first aspect or any possible implementation of the first aspect.

[0111] The seventh aspect and any implementation of the seventh aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the seventh aspect and any implementation of the seventh aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0112] In an eighth aspect, the present application provides a chip comprising one or more interface circuits and one or more processors; the interface circuit is used to receive signals from a memory of an electronic device and send signals to the processor, the signals comprising computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the method of the fourth aspect or any possible implementation of the fourth aspect.

[0113] The eighth aspect and any implementation of the eighth aspect correspond to the fourth aspect and any implementation of the fourth aspect, respectively. The technical effects corresponding to the eighth aspect and any implementation of the eighth aspect can be referred to the technical effects corresponding to the fourth aspect and any implementation of the fourth aspect, and will not be repeated here.

[0114] In a ninth aspect, the present application provides a computer-readable storage medium storing a computer program. When the computer program runs on a computer or a processor, the computer or the processor executes the method in the first aspect or any possible implementation of the first aspect.

[0115] The ninth aspect and any implementation of the ninth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the ninth aspect and any implementation of the ninth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0116] In the tenth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer or a processor, it enables the computer or the processor to execute the method in the fourth aspect or any possible implementation of the fourth aspect.

[0117] The tenth aspect and any implementation of the tenth aspect correspond to the fourth aspect and any implementation of the fourth aspect, respectively. The technical effects corresponding to the tenth aspect and any implementation of the tenth aspect can be referred to the technical effects corresponding to the fourth aspect and any implementation of the fourth aspect, and will not be repeated here.

[0118] In an eleventh aspect, the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed by a computer or a processor, the computer or the processor executes the method in the first aspect or any possible implementation of the first aspect.

[0119] The eleventh aspect and any implementation of the eleventh aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the eleventh aspect and any implementation of the eleventh aspect can be referred to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, and will not be repeated here.

[0120] In a twelfth aspect, the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed by a computer or a processor, the computer or the processor executes the method in the fourth aspect or any possible implementation of the fourth aspect.

[0121] The twelfth aspect and any implementation of the twelfth aspect correspond to the fourth aspect and any implementation of the fourth aspect, respectively. The technical effects corresponding to the twelfth aspect and any implementation of the twelfth aspect can be referred to the technical effects corresponding to the above-mentioned fourth aspect and any implementation of the fourth aspect, and will not be repeated here.

[0122] In a thirteenth aspect, the present application provides a computer-readable storage medium, which stores an encoded code stream in the second aspect (or the third aspect) or any possible implementation of the second aspect (or the third aspect).

[0123] The thirteenth aspect and any one of the implementations of the thirteenth aspect correspond to the second aspect (or the third aspect) and any one of the implementations of the second aspect (or the third aspect), respectively. The technical effects corresponding to the thirteenth aspect and any one of the implementations of the thirteenth aspect can be referred to the technical effects corresponding to the above-mentioned second aspect (or the third aspect) and any one of the implementations of the second aspect (or the third aspect), and will not be repeated here.

[0124] In a fourteenth aspect, the present application provides a device for storing a code stream, the device comprising: a receiver and at least one storage medium, the receiver being used to receive an encoded code stream; the at least one storage medium being used to store the encoded code stream; the encoded code stream being generated according to the second aspect (or the third aspect) and any one of the implementation methods of the second aspect (or the third aspect).

[0125] The fourteenth aspect and any implementation of the fourteenth aspect correspond to the second aspect (or the third aspect) and any implementation of the second aspect (or the third aspect), respectively. The technical effects corresponding to the fourteenth aspect and any implementation of the fourteenth aspect can be referred to the technical effects corresponding to the above-mentioned second aspect (or the third aspect) and any implementation of the second aspect (or the third aspect), and will not be repeated here.

[0126] In the fifteenth aspect, the present application provides a device for transmitting a code stream, the device comprising: a transmitter and at least one storage medium, the at least one storage medium being used to store an encoded code stream, the encoded code stream being generated according to the second aspect (or the third aspect) and any one of the implementation methods of the second aspect (or the third aspect); the transmitter being used to obtain the encoded code stream from the storage medium and send the encoded code stream to the end-side device through the transmission medium.

[0127] The fifteenth aspect and any implementation of the fifteenth aspect correspond to the second aspect (or the third aspect) and any implementation of the second aspect (or the third aspect), respectively. The technical effects corresponding to the fifteenth aspect and any implementation of the fifteenth aspect can be referred to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, and will not be repeated here.

[0128] In a sixteenth aspect, the present application provides a system for distributing code streams, the system comprising: at least one storage medium for storing at least one encoded code stream, the at least one encoded code stream being generated according to the second aspect (or the third aspect) and any one of the implementation methods of the second aspect (or the third aspect), a streaming media device for obtaining a target code stream from the at least one storage medium and sending the target code stream to an end-side device, wherein the streaming media device includes a content server or a content distribution server.

[0129] The sixteenth aspect and any implementation of the sixteenth aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the sixteenth aspect and any implementation of the sixteenth aspect can be referred to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, and will not be repeated here.

[0130] In a seventeenth aspect, the present application provides an encoding device, which may include:

[0131] A fitting processing module is used to perform fitting processing on the pixel values ​​of multiple texture pixels in the texture image to obtain a geometric surface;

[0132] A projection point acquisition module is used to obtain the coordinates of multiple projection points in the geometric surface; wherein the multiple projection points correspond one-to-one to the pixel values ​​of the multiple texture pixels, and the coordinates of the projection points include two or more dimensions;

[0133] An endpoint acquisition module is used to obtain values ​​representing three or more endpoints of a geometric surface;

[0134] The encoding module is used to encode the values ​​of three or more endpoints and the coordinates of multiple projection points to obtain an encoded code stream.

[0135] It should be understood that the encoding device of the seventeenth aspect can execute the method steps in the first aspect or any implementation of the first aspect.

[0136] The seventeenth aspect and any implementation of the seventeenth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the seventeenth aspect and any implementation of the seventeenth aspect can be referred to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, and will not be repeated here.

[0137] In an eighteenth aspect, the present application provides a decoding device, which may include:

[0138] A code stream acquisition module, used to obtain the encoded code stream;

[0139] a reconstruction information acquisition module, configured to decode the encoded bitstream to obtain reconstruction values ​​of three or more endpoints and coordinate reconstruction values ​​of a plurality of projection points; wherein the three or more endpoints are used to represent a geometric surface, the geometric surface includes a plurality of projection points, the plurality of projection points correspond one-to-one to pixel values ​​of a plurality of texture pixels in the texture image, and the coordinate reconstruction values ​​include two or more dimensions;

[0140] The reconstruction value generating module is used to determine the reconstruction values ​​of multiple texture pixels in the texture image according to the reconstruction values ​​of three or more endpoints and the coordinate reconstruction values ​​of multiple projection points.

[0141] It should be understood that the decoding device of the eighteenth aspect can execute the method steps in the fourth aspect or any implementation of the fourth aspect.

[0142] The eighteenth aspect and any implementation of the eighteenth aspect correspond to the fourth aspect and any implementation of the fourth aspect, respectively. The technical effects corresponding to the eighteenth aspect and any implementation of the eighteenth aspect can be referred to the technical effects corresponding to the fourth aspect and any implementation of the fourth aspect, and will not be repeated here.

[0143] In a nineteenth aspect, the present application provides an encoder comprising a processing circuit for executing the method steps in the first aspect and any one of the implementations of the first aspect.

[0144] The nineteenth aspect and any implementation of the nineteenth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the nineteenth aspect and any implementation of the nineteenth aspect can be referred to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, and will not be repeated here.

[0145] In the twentieth aspect, the present application provides a decoder comprising a processing circuit for executing the method steps in the fourth aspect and any one of the implementations of the fourth aspect.

[0146] The twentieth aspect and any implementation of the twentieth aspect correspond to the fourth aspect and any implementation of the fourth aspect, respectively. The technical effects corresponding to the twentieth aspect and any implementation of the twentieth aspect can be referred to the technical effects corresponding to the fourth aspect and any implementation of the fourth aspect, and will not be repeated here.

[0147] In a twenty-first aspect, the present application provides a device for transmitting a code stream, the device comprising: a transmitter and a receiver,

[0148] A receiver configured to receive an encoded code stream generated according to the second aspect (or the third aspect) and any one of the implementations of the second aspect (or the third aspect);

[0149] The transmitter is used to send the encoded code stream to the end-side device through a transmission medium.

[0150] The twenty-first aspect and any implementation of the twenty-first aspect correspond to the second aspect (or the third aspect) and any implementation of the second aspect (or the third aspect), respectively. The technical effects corresponding to the twenty-first aspect and any implementation of the twenty-first aspect can be referred to the technical effects corresponding to the second aspect (or the third aspect) and any implementation of the second aspect (or the third aspect), and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0151] FIG1A is a schematic diagram of a framework of an exemplary system 100;

[0152] FIG1B is a schematic diagram illustrating an exemplary application scenario;

[0153] FIG1C is a schematic diagram illustrating an exemplary application scenario;

[0154] FIG2A is a schematic diagram showing the structure of an encoder;

[0155] FIG2B is a schematic diagram illustrating an exemplary encoding process of a texture image;

[0156] FIG2C is a schematic diagram illustrating an exemplary encoding process of a texture image;

[0157] FIG2D is a schematic diagram illustrating an exemplary encoding process of a texture image;

[0158] FIG3A is a schematic diagram illustrating an exemplary texture image encoding process 300;

[0159] FIG3B is a schematic diagram illustrating an exemplary fitting process and a projection process;

[0160] FIG4A is a schematic structural diagram of an exemplary decoder;

[0161] FIG4B is a schematic diagram illustrating an exemplary decoding process of a texture image;

[0162] FIG4C is a schematic diagram illustrating an exemplary decoding process of a texture image;

[0163] FIG4D is a schematic diagram illustrating an exemplary decoding process of a texture image;

[0164] FIG5A is a schematic diagram illustrating an exemplary texture image decoding process 500;

[0165] FIG5B is a schematic diagram illustrating an exemplary rendering process and a texture image decoding process;

[0166] FIG6A is a schematic diagram illustrating an exemplary texture image encoding process 600;

[0167] FIG6B is a schematic diagram illustrating an exemplary process of selecting a coding mode combination with the smallest coding error;

[0168] FIG7A is a schematic diagram illustrating an exemplary structure of an encoded bitstream;

[0169] FIG7B is a schematic diagram illustrating an exemplary structure of an encoded code stream;

[0170] FIG7C is a schematic diagram illustrating an exemplary structure of an encoded bitstream;

[0171] FIG8 is a schematic diagram illustrating an exemplary texture image decoding process 800;

[0172] FIG9 is a schematic diagram illustrating an exemplary texture image encoding process 900;

[0173] FIG10 is a schematic diagram showing an exemplary projection point of a texture pixel onto a line and a geometric surface;

[0174] FIG11 is a schematic diagram of an exemplary texture image encoding apparatus 1100;

[0175] FIG12 is a schematic diagram illustrating an exemplary texture image decoding apparatus 1200;

[0176] FIG13 is a schematic structural diagram of an exemplary device. DETAILED DESCRIPTION

[0177] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0178] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0179] In the description and claims of the embodiments of this application, the terms "first" and "second" are used to distinguish different objects, rather than to describe a specific order of objects. For example, the terms "first target object" and "second target object" are used to distinguish different objects, rather than to describe a specific order of objects.

[0180] In the embodiments of this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a concrete manner.

[0181] In the description of the embodiments of this application, unless otherwise specified, "multiple" means two or more. For example, "multiple processing units" means two or more processing units; "multiple systems" means two or more systems.

[0182] To facilitate understanding by those skilled in the art, the relationship between the compression process (encoding process and decoding process) and the rendering process of a texture image is first introduced.

[0183] FIG1A is a schematic diagram showing a framework of an exemplary system 100 .

[0184] 1A , for example, during the development phase of a 3D application, a 3D model 101, material information for the 3D model 101, and a first texture image for the 3D model 101 may be generated. The pixel values ​​of the first texture image represent color information (e.g., RGB, YUV, etc., and multiple first texture images may be provided). Subsequently, the material information may be saved as a second texture image. The pixel values ​​of the second texture image represent material information, and multiple second texture images may be provided.

[0185] For example, the material information may include, but is not limited to, roughness, normal information, ambient light information, etc., which is not limited in this application. Correspondingly, the pixel values ​​of the second texture image may be roughness, normal information, ambient light information, etc., which is not limited in this application.

[0186] Hereinafter, the first texture image and the second texture image are collectively referred to as texture image 102 .

[0187] For example, each texture image 102 may include at least two channels, and the pixel value of a texture pixel in the texture image may include at least two components. It should be noted that the number of components included in the pixel value of a texture pixel is the same as the number of channels included in the texture image; and the multiple components included in the pixel value of a texture pixel correspond one-to-one to the multiple channels of the texture image.

[0188] For example, a texture image includes three channels (R channel, G channel, and B channel), and correspondingly, a pixel value of a texture pixel includes three components: R component, G component, and B component; in this case, the texture image can be called an RGB image.

[0189] For another example, a texture image includes two channels (a roughness channel and an ambient light channel), and correspondingly, a pixel value of a texture pixel includes two components: a roughness component and an ambient light component.

[0190] For another example, a texture image includes four channels (R channel, G channel, B channel and A (transparency) channel), and correspondingly, the pixel value of a texture pixel may include four components: R component, G component, B component and A component.

[0191] Continuing with FIG. 1A , illustratively, texture image 102 may be input to encoder 110 (also referred to as a texture image encoder), which encodes texture image 102 to generate encoded bitstream 104 (also referred to as a texture image bitstream). Furthermore, geometric information of 3D model 101 may be represented using mesh 103 . It should be understood that the geometric information of 3D model 101 may also be represented using other forms, such as point clouds, implicit representations (e.g., neural networks), and the present application does not limit this.

[0192] It should be noted that the grid 103 may also be encoded to obtain a grid code stream, and this application does not impose any limitation on this.

[0193] Continuing with reference to FIG1A , illustratively, during the usage phase of a 3D application, when rendering is required, the encoded code stream 104 and the mesh 103 can be loaded into the memory 111. Then, the mesh 103 can be loaded from the memory 111 into the renderer 114 of the GPU (112). The renderer 114 can start the rendering pipeline, which performs rendering processing (Rendering) based on the mesh 103 to generate a rendered image 106. During the rendering process of the rendering pipeline, when the texture image 102 is needed, the encoded code stream 104 can be loaded from the memory 111 into the decoder 113 (also referred to as a texture image decoder). The decoder 113 decodes the encoded code stream 104 to obtain the decoded texture image 105 and outputs the decoded texture image 105 to the renderer 114 for use in the rendering process of the rendering pipeline.

[0194] It should be noted that, in order to reduce the bandwidth between the memory 111 and the GPU ( 112 ), the encoding of the texture image 102 by the encoder 110 may be lossy encoding, and thus, there are differences between the texture image 102 and the decoded texture image 105 .

[0195] In one possible embodiment, the system 100 may include an electronic device, that is, the encoder 110, the memory 111 and the GPU (112) belong to the same electronic device.

[0196] In one possible embodiment, the system 100 may include two electronic devices, wherein the encoder 110 is located in one electronic device, and the memory 111 and the GPU ( 112 ) are located in another electronic device.

[0197] It should be noted that although FIG1A shows that the decoder 113 is disposed inside the GPU (112), it should be understood that the decoder 113 may also be disposed outside the GPU (112). When the decoder 113 is disposed outside the GPU (112), if the system 100 includes two electronic devices, the encoder 110 is located in one electronic device, and the decoder 113, the memory 111, and the GPU (112) are located in the other electronic device.

[0198] It should be understood that the decoder 113 can be implemented by hardware (such as hardening in a GPU to improve decoding efficiency and thus rendering efficiency), or by software (such as by a CPU), and this application does not limit this. The encoder 110 can be implemented by hardware or by software, and this application does not limit this. It should be understood that the encoder 110 and decoder 113 can also be implemented by other general-purpose chips or dedicated chips, and this application does not limit this.

[0199] Exemplarily, 3D applications may include, but are not limited to, cloud gaming, virtual reality (VR) / augmented reality (AR) applications (such as AR / VR conferences), digital human live broadcasts, digital museums, virtual concerts, etc., and this application does not impose any restrictions on this.

[0200] FIG1B is a schematic diagram illustrating an exemplary application scenario. In FIG1B , system 100 includes server 120 . In other words, server 120 performs encoding and decoding (including encoding and decoding) and rendering of texture images. The terminal device in FIG1B is used to display the rendered image.

[0201] Referring to Figure 1B , for example, the encoder in server 120 can compress the texture images required for cloud gaming to obtain and store an encoded bitstream. When a user uses mobile phone 131 to play cloud gaming and needs to render, the renderer in server 120 can perform the rendering process to obtain the game screen (i.e., the rendered game screen). During the rendering process, the decoder is called to decode the encoded bitstream and obtain the decoded texture image for the rendering process. The game screen can then be sent to mobile phone 131 via the network, and mobile phone 131 can display the game screen.

[0202] Referring to Figure 1B , for example, the encoder in server 120 can compress the texture images required for an AR / VR conference to obtain and store an encoded bitstream. When rendering is required during an AR / VR conference, the renderer in server 120 can perform the rendering process to obtain a conference screen (i.e., a rendered conference screen). During the rendering process, the decoder is called to decode the encoded bitstream and obtain the decoded texture image for use in the rendering process. The conference screen can then be sent over the network to VR / AR glasses 132, which can then display the conference screen.

[0203] Referring to FIG1B , for example, the encoder in server 120 can compress the texture images required for the virtual concert to generate and store an encoded bitstream. When rendering is required while the user is watching the virtual concert, the renderer in server 120 can perform the rendering process to generate the concert image (i.e., the rendered concert image). During the rendering process, the decoder can be called to decode the encoded bitstream and obtain the decoded texture image for use in the rendering process. The concert image can then be sent via the network to personal computer 133, which can then display the concert image.

[0204] Referring to Figure 1B , for example, the encoder in server 120 can compress the texture image required for live broadcast to obtain an encoded bitstream and store it. When rendering is required while the user is watching the live broadcast of the digital human, the renderer in server 120 can perform the rendering process to obtain a rendered image, namely the live broadcast screen (i.e., the rendered live broadcast screen). During the rendering process, the decoder is called to decode the encoded bitstream and obtain the decoded texture image for the rendering process. The live broadcast screen can then be sent to tablet computer 134 via the network, which can then display the live broadcast screen.

[0205] Figure 1C is a schematic diagram illustrating an exemplary application scenario. In Figure 1C, system 100 includes a server 120 and a terminal device (including at least one of the following: a mobile phone 131, VR / AR glasses 132, a personal computer 133, and a tablet computer 134, etc., which are not limited in this application). The server 120 compresses the texture image, and the terminal device decompresses the texture image, renders it, and displays the rendered image.

[0206] 1C , exemplarily, the encoder in the server 120 can compress the texture image required for the cloud game to obtain an encoded code stream and store it. When the user uses the mobile phone 131 to play the cloud game, when rendering is required, the server 120 can send the encoded code stream to the mobile phone 131 through the network. Subsequently, the renderer in the mobile phone 131 can perform the rendering process (wherein the grid 103 required for rendering can be sent to the mobile phone 131 by the server 120 during the cloud game, or it can be obtained from the server 120 or a third-party device by the mobile phone 131 before the cloud game starts. This application is not limited to this) to obtain the game screen (i.e., the rendered game screen); and call the decoder during the rendering process to decode the encoded code stream and obtain the decoded texture image for the rendering process. After obtaining the game screen, the mobile phone 131 can display the game screen.

[0207] 1C , illustratively, the encoder in server 120 can compress the texture image required for the VR / AR conference to obtain and store an encoded bitstream. When a user uses AR / VR glasses 132 to play a VR / AR conference and rendering is required, server 120 can send the encoded bitstream to the VR / AR glasses 132 via the network. Subsequently, the renderer in VR / AR glasses 132 can perform rendering processing (wherein the mesh 103 required for rendering can be sent by server 120 to VR / AR glasses 132 during the VR / AR conference, or obtained by VR / AR glasses 132 from server 120 or a third-party device before the VR / AR conference begins, and this application is not limited to this), obtaining a conference screen (i.e., a rendered conference screen); and during the rendering process, calling a decoder to decode the encoded bitstream, obtaining a decoded texture image for use in the rendering process. After obtaining the conference screen, VR / AR glasses 132 can display the conference screen.

[0208] 1C , exemplarily, the encoder in the server 120 can compress the texture image required for the virtual concert to obtain and store the encoded bitstream. When the user uses the personal computer 133 to play the virtual concert and needs to render, the server 120 can send the encoded bitstream to the personal computer 133 via the network; the renderer in the personal computer 133 can perform the rendering process (wherein the grid 103 required for rendering can be sent by the server 120 to the personal computer 133 during the virtual concert, or can be obtained by the personal computer 133 from the server 120 or a third-party device before the virtual concert begins, and this application does not limit this) to obtain the concert picture (rendered concert picture); and during the rendering process, the decoder is called to decode the encoded bitstream to obtain the decoded texture image for the rendering process. After obtaining the concert picture, the personal computer 133 can display the concert picture.

[0209] 1C , illustratively, the encoder in server 120 can compress the texture image required for live broadcast to obtain and store an encoded bitstream. While a user is watching the live broadcast of the digital human on tablet computer 134, server 120 can send the encoded bitstream to tablet computer 134 via the network. The renderer in tablet computer 134 can perform rendering processing (wherein the mesh 103 required for rendering can be sent by server 120 to tablet computer 134 during the live broadcast, or obtained by tablet computer 134 from server 120 or a third-party device before the live broadcast begins, and this application does not impose any restrictions on this), obtaining a live broadcast image (i.e., a rendered live broadcast image). During the rendering process, the decoder is called to decode the encoded bitstream and obtain a decoded texture image for use in the rendering process. After obtaining the live broadcast image, tablet computer 134 can display the live broadcast image.

[0210] It should be noted that the terminal device can also encode and decode the texture image, perform rendering processing, and display the rendered image, and this application does not impose any restrictions on this.

[0211] The following describes the encoding and decoding process of texture images.

[0212] Fig. 2A is a schematic diagram of the structure of an encoder. Fig. 2A shows an example of the encoder 110 in Fig. 1A.

[0213] 2A , illustratively, the encoder 110 may include a splitting module 201, a fitting processing module 202, a transform module 203, a first quantization module 204, a first encoding module 205, a pre-processing module 206, a second quantization module 207, and a second encoding module 208. The splitting module 201, the transform module 203, and the pre-processing module 206 are optional modules.

[0214] It should be noted that the first quantization module 204 and the second quantization module 207 can be combined into one module, and the first encoding module 205 and the second encoding module 208 can be combined into one module, which is not limited in this application.

[0215] It should be understood that the encoder 110 of the present application may include more or fewer modules than those shown in FIG. 2A , and the present application does not limit this.

[0216] The following describes the encoding process of the texture image with reference to FIG. 2B to FIG. 2D based on FIG. 2A .

[0217] FIG2B is a schematic diagram illustrating an exemplary encoding process of a texture image.

[0218] 2A and 2B , the encoding process of a texture image may be as follows: a fitting processing module 202 may perform fitting processing 21 on the texture image 102 to obtain the coordinates 120 of the projection point corresponding to the pixel value of each texture pixel in the texture image 102. Next, a transformation module 203 may perform a transformation (e.g., discrete cosine transform (DCT), discrete wavelet transform (DWT), singular value decomposition (SVD), dimensionality reduction, etc.) 27 on the coordinates 120 corresponding to the pixel value of the projection point to obtain a transformed coordinate value 128 of the pixel value of the projection point. Subsequently, a first quantization module 204 may perform quantization processing (also referred to as quantization) 22 on the transformed coordinate value 128 of the pixel value of the projection point to obtain a quantized coordinate value 121 of the pixel value of the projection point. Thereafter, a first encoding module 205 may encode 23 the quantized coordinate value 121 of the pixel value of the projection point to obtain an encoded bitstream 104.

[0219] FIG2C is a schematic diagram illustrating an exemplary encoding process of a texture image.

[0220] 2A and 2C , the encoding process for a texture image may be exemplarily as follows: a splitting module 201 may perform a splitting process (also referred to as splitting or segmenting) 24 on a texture image 102 to obtain a plurality of texture blocks. Next, a fitting module 202 may perform a fitting process 21 on each texture block 122 to obtain the coordinates 120 of the projection point corresponding to the pixel value of each texture pixel in each texture block 122. Next, a transformation module 203 may perform a transformation 27 on the coordinates 120 corresponding to the projection point of the pixel value to obtain a transformed value 128 of the coordinates corresponding to the projection point of the pixel value. Subsequently, a first quantization module 204 may perform a quantization process (also referred to as quantization) 22 on the transformed value 128 of the coordinates corresponding to the projection point of the pixel value to obtain a quantized value 121 of the coordinates corresponding to the projection point of the pixel value. Thereafter, a first encoding module 205 may encode 23 the quantized value 121 of the coordinates corresponding to the projection point of the pixel value to obtain a plurality of codestream blocks 123; wherein one texture block 122 is encoded into one codestream block 123. The plurality of code stream blocks 123 may then be concatenated 25 to obtain an encoded code stream 104 .

[0221] It should be noted that the encoding process shown in FIG. 2C may not perform splicing 25 . In this case, each obtained code stream block 123 may be sent or stored.

[0222] It should be noted that splitting 24 and splicing 25 are optional steps in the texture image encoding process.

[0223] FIG2D is a schematic diagram illustrating an exemplary encoding process of a texture image.

[0224] 2A and 2D , the encoding process for a texture image may be exemplarily as follows: a splitting module 201 may perform a splitting process (also referred to as splitting or segmenting) 24 on a texture image 102 to obtain a plurality of texture blocks 122. Next, the splitting module 201 may further split 26 each texture block 122 to obtain a plurality of regions 124; each texture block 122 is split into a plurality of regions 124. Subsequently, a fitting module 202 may perform a fitting process 21 on each region 124 to obtain coordinates 120 of a projection point corresponding to the pixel value of each texture pixel in each region 124. Subsequently, a transformation module 203 may perform a transformation 27 on the coordinates 120 of the projection point of each texture pixel in the texture image 102 to obtain a transformed value 128 of the coordinates of the projection point corresponding to the pixel value. Subsequently, a first quantization module 204 may perform a quantization process (also referred to as quantization) 22 on the coordinates 120 of the projection point corresponding to the pixel value to obtain a quantized value 121 of the coordinates of the projection point corresponding to the pixel value. Afterwards, the first encoding module 205 may encode 23 the coordinate quantized value 121 of the projection point corresponding to the pixel value to obtain a plurality of code stream blocks 123, wherein each region 124 is encoded into a code stream block 123. The plurality of code stream blocks 123 may then be concatenated 25 to obtain an encoded code stream 104.

[0225] It should be noted that the encoding process shown in FIG. 2D may not perform the splicing 25 . In this case, each obtained code stream block 123 may be sent or stored.

[0226] It should be noted that splitting 24, splitting 26 and splicing 25 are optional steps in the encoding process of the texture image.

[0227] Referring to Figure 2A again, the fitting processing module 202 performs fitting processing on the texture image 102 and can also obtain values ​​used to represent the endpoints of the fitting surface; the values ​​used to represent the endpoints of the fitting surface (such as a geometric surface) can be regarded as auxiliary information, which is used to assist in reconstructing the texture image 102. Afterwards, the pre-processing module 206 can perform pre-processing (also known as pre-processing, such as residual processing, etc.) on the values ​​of the endpoints to obtain pre-processed values ​​of the endpoints. Next, the second quantization module 207 can perform quantization processing on the pre-processed values ​​of the endpoints to obtain quantized values ​​of the endpoints. Subsequently, the second encoding module 208 can encode the quantized values ​​of the endpoints to obtain an encoded bitstream. It should be noted that pre-processing (or pre-processing) is an optional step.

[0228] FIG3A is a schematic diagram illustrating an exemplary texture image encoding process 300 .

[0229] S301 , performing fitting processing on pixel values ​​of a plurality of texture pixels in a texture image to obtain a geometric surface.

[0230] FIG3B is a schematic diagram illustrating an exemplary fitting process and a projection process.

[0231] For example, a fitting point can be determined based on the pixel value of a texel in a texture image; the coordinates of the fitting point are the pixel value of the texel; specifically, a component of the pixel value of the texel can be used as a dimension of the coordinates of the fitting point. Referring to FIG3B , the pixel value (R1, G1, B1) of texel P1 corresponds to a fitting point A with coordinates (R1, G1, B1); the pixel value (R2, G2, B2) of texel P2 corresponds to a fitting point B with coordinates (R2, G2, B2). In other words, the pixel value of each texel in a texture image can be used to determine a fitting point; for example, if a texture image includes 64*64 texels, 64*64 fitting points can be determined.

[0232] The dimension of the coordinates of the fitting points is the same as the number of components contained in the pixel value of the texture pixel (or the number of channels of the texture image).

[0233] For example, a data optimization method can be used to fit the pixel values ​​of multiple texture pixels in a texture image to obtain an objective function, which is used to represent a geometric surface in space; that is, by fitting the pixel values ​​of multiple texture pixels in a texture image, a geometric surface can be obtained. For example, the geometric surface of this application can include any of the following types: plane, curved surface, and geometric body. The geometric surface in Figure 3B is a plane.

[0234] It should be noted that this application does not limit the data optimization method used in the fitting process. For example, it can be principal component analysis (PCA), least squares method, gradient descent method, etc. The specific fitting process can be referred to the description in the prior art and will not be repeated here.

[0235] For example, when the pixel value of a texel includes two components, the pixel value of each texel may correspond to a fitting point in two-dimensional space. For example, if the texture image is a normal vector texture map, the pixel value of each texel may correspond to a fitting point with coordinates (F1, F2) in two-dimensional space (F1 and F2 are two normal vectors). Accordingly, the geometric surface obtained in S301 is a surface in two-dimensional space.

[0236] For example, when the pixel value of a texture pixel includes three components, the pixel value of each texture pixel may correspond to a fitting point in three-dimensional space. For example, if the texture image is an RGB image, the pixel value of each texture pixel may correspond to a fitting point with coordinates (R, G, B) in three-dimensional space. Accordingly, the geometric surface obtained in S301 is a surface in three-dimensional space.

[0237] For example, when the pixel value of a texture pixel includes four components, the pixel value of each texture pixel may correspond to a fitting point in four-dimensional space. For example, if the pixel value of a texture image is color information (RGB) and transparency, the pixel value of each texture pixel may correspond to a fitting point with coordinates (R, G, B, A) (A represents transparency) in four-dimensional space. Accordingly, the geometric surface obtained in S301 is a surface in four-dimensional space.

[0238] The same goes for other cases, so I won’t go into details here.

[0239] In one possible approach, encoding can be performed using a texture image as a unit, that is, executing S301 to S304. In other words, the pixel values ​​of all texture pixels included in the texture image can be fitted to obtain a geometric surface.

[0240] In one possible approach, a texture image can be split into multiple texture blocks, each of which can include multiple texture pixels. Then, encoding is performed on each texture block (i.e., encoding is performed on a texture block basis), i.e., steps S301 to S304 are performed. In other words, S301 can be performing a fitting process on the pixel values ​​of all texture pixels in any texture block of the texture image to obtain a geometric surface.

[0241] In one possible approach, a texture image can be split into multiple texture blocks; each texture block can then be split into multiple areas; each area can include multiple texture pixels; and each area can then be encoded (i.e., encoded in units of areas), i.e., S301 to S304 can be performed. In other words, S301 can be fitting the pixel values ​​of all texture pixels in any area of ​​the texture image to obtain a geometric surface.

[0242] S302, obtaining coordinates of a plurality of projection points in a geometric surface; wherein the plurality of projection points correspond one-to-one to pixel values ​​of a plurality of texture pixels, and the coordinates of the projection points include two or more dimensions.

[0243] For example, multiple fitting points (coordinates are pixel values ​​of texture pixels) can be projected onto the geometric surface to determine multiple projection points. For example, one fitting point corresponds to one projection point (which can also be understood as one pixel value corresponding to one projection point). The coordinates of the projection points can be two-dimensional or more.

[0244] Exemplarily, when the geometric surface is a plane or a curved surface, the coordinates of the projection point may include two dimensions.

[0245] Referring again to Figure 3B , the geometric surface in Figure 3B is a plane, and the coordinates of the projection points of the fitting points onto the geometric surface include two dimensions. In Figure 3B , the black points are the fitting points, and the white points are the projection points of the fitting points on the plane. For example, the projection point of fitting point A (R1, G1, B1) on the plane is A1, and the coordinates of projection point A1 are (u1, v1). The coordinates of projection point A1 include two dimensions.

[0246] Exemplarily, when the geometric surface is a geometric body, the coordinates of the projection points of the fitting points onto the geometric surface may include more than two dimensions.

[0247] S303: Obtain values ​​used to represent three or more endpoints of the geometric surface.

[0248] Exemplarily, the values ​​of three or more endpoints that can be used to represent a geometric surface can be obtained, and then the values ​​of the three or more endpoints used to represent the geometric surface can be transmitted to the decoding end; in this way, the decoder can combine the values ​​of the three or more endpoints used to represent the geometric surface and the coordinates of the projection point of the fitting point to the geometric surface to restore (reconstruct) the pixel value of the texture pixel (which can be called a reconstructed value).

[0249] For example, when the geometric surface is a plane or a curved surface, three endpoints for representing the plane or the curved surface may be obtained, as shown by ept0, ept1, and ept2 in FIG3B .

[0250] Exemplarily, when the geometric surface is a geometric body, four endpoints for representing the geometric body may be obtained.

[0251] For example, each endpoint may include multiple components, and the number of components included in each endpoint may be the same as the number of channels included in the texture image.

[0252] It should be noted that this application does not limit the execution order of S302 and S303.

[0253] It should be noted that S301 to S303 may be executed by the fitting processing module 202 in FIG. 2A .

[0254] S304 , encoding the values ​​of three or more endpoints and encoding the coordinates of multiple projection points to obtain an encoded code stream.

[0255] For example, the first encoding module 205 may encode the coordinates of a plurality of projection points, and the second encoding module 208 may encode the values ​​representing three or more endpoints of a geometric surface to obtain an encoded code stream.

[0256] Referring again to FIG. 2A , illustratively, before executing S304 , the first quantization module 204 may further quantize the coordinates of the projection point to obtain a quantized value of the coordinates of the projection point; thereafter, the first encoding module 205 may encode the quantized value of the coordinates of the projection point. Furthermore, the second quantization module 207 may further quantize the values ​​of three or more endpoints used to represent the geometric surface to obtain quantized values ​​of the three or more endpoints used to represent the geometric surface; thereafter, the second encoding module 208 may encode the quantized values ​​of the three or more endpoints used to represent the geometric surface.

[0257] Referring to FIG. 2A again, illustratively, before the first quantization module 204 quantizes the coordinates of the projection point, the transformation module 203 may further transform the coordinates of the projection point; specifically, reference may be made to the description of FIG. 2C and FIG. 2D .

[0258] For example, before the second quantization module 207 quantizes the values ​​of the three or more endpoints used to represent the geometric surface, the pre-processing module 206 may further pre-process the values ​​of the three or more endpoints used to represent the geometric surface.

[0259] It should be noted that, assuming that the three or more endpoints used to represent the geometric surface include a first portion of endpoints and a second portion of endpoints, the values ​​of the first portion of endpoints can be preprocessed and then quantized and encoded; or the values ​​of the second portion of endpoints can be directly quantized and encoded.

[0260] Referring again to FIG. 3B , illustratively, after the coordinates (u1, v1) of the projection point A1 of the fitting point A on the plane are quantized, A2(u2, v2) is obtained, as shown by the gray point in the plane of FIG. 3B .

[0261] It should be noted that the positions of A2 (u2, v2) and A1 (u1, v1) may or may not overlap, and this application does not impose any restrictions on this.

[0262] Exemplarily, the encoding algorithms used by the first encoding module 205 and the second encoding module 208 include but are not limited to: entropy coding, ASCII coding, BISE (Bounded integer sequence encoding) mechanism provided by ASTC (Adaptive Scalable Texture Compression), etc., which are not limited in this application.

[0263] Exemplarily, when the texture pixels used for fitting processing in S301 are all the texture pixels in the texture image, after executing S304 , the encoded bitstream obtained is the bitstream of the texture image.

[0264] Exemplarily, when the texture pixels used for fitting processing in S301 are all the texture pixels in a texture block, after executing S304, the encoded code stream obtained is the code stream block of the texture block; thereafter, S301 to S304 can be executed for another texture block in the texture image to obtain the code stream block of another texture block; and so on, S301 to S304 are executed in a loop until all texture blocks in the texture image are encoded; in this way, multiple code stream blocks can be obtained; and multiple code stream blocks can be spliced ​​to obtain the code stream of the texture image.

[0265] For example, if the texture pixels used for fitting in S301 are all the texture pixels in a region, then after executing S304, the encoded bitstream obtained is a bitstream block for the region. S301 to S304 can then be executed for another region in the texture block to obtain a bitstream block for another region. S301 to S304 can be executed repeatedly until all regions in the texture block are encoded. In this way, multiple bitstream blocks can be obtained. Subsequently, encoding can be performed for another texture block in the same manner to obtain multiple bitstream blocks. This can be repeated until all texture blocks in the texture image are encoded. The multiple bitstream blocks can then be concatenated to obtain a bitstream for the texture image.

[0266] Fig. 4A is a schematic diagram of the structure of a decoder. Fig. 4A shows an example of the decoder 113 in Fig. 1A.

[0267] 4A , illustratively, the decoder 113 may include a first decoding module 401, a first inverse quantization module 402, an inverse transform module 403, a second decoding module 404, a second inverse quantization module 405, a post-processing module 406, and an interpolation module 407. The inverse transform module 403 and the post-processing module 406 are optional modules.

[0268] It should be noted that the first inverse quantization module 402 and the second inverse quantization module 405 can be combined into one module, and the first decoding module 401 and the second decoding module 404 can be combined into one module, which is not limited in this application.

[0269] It should be understood that the decoder 113 of the present application may include more or fewer modules than those shown in FIG. 4A , and the present application does not impose any limitation thereto.

[0270] The following describes the decoding process of the texture image with reference to FIG. 4B to FIG. 4D based on FIG. 4A .

[0271] FIG4B is a schematic diagram of an exemplary decoding process of a texture image, wherein the decoding process of FIG4B corresponds to the encoding process of FIG2B .

[0272] 4A and 4B , the decoding process of a texture image may be exemplarily as follows: a first decoding module 401 may decode 41 the encoded bitstream 104 to obtain a quantized coordinate value 121 corresponding to the projection point of the pixel value. Next, a first dequantization module 402 may dequantize (or dequantize) 42 the quantized coordinate value 121 corresponding to the projection point of the pixel value, obtaining a dequantized coordinate value 127 corresponding to the projection point of the pixel value. Subsequently, an inverse transformation module 403 may perform an inverse transformation (or inverse transformation; for example, inverse DCT, inverse DWT, inverse SVD, etc.) 46 on the dequantized coordinate value 127 corresponding to the projection point of the pixel value, obtaining a reconstructed coordinate value 129 corresponding to the projection point of the pixel value. Subsequently, an interpolation module 407 may perform interpolation 43 based on the reconstructed coordinate value 129 corresponding to the projection point of the pixel value, obtaining a decoded texture image 105.

[0273] It should be noted that the coordinates 120 of the projection point corresponding to the pixel value and the coordinate reconstructed value 129 of the projection point corresponding to the pixel value may be the same or different, and this application does not impose any limitation on this.

[0274] Here, for example, the pixel values ​​of the texels in texture image 104 are distinguished from the pixel values ​​of the texels in decoded texture image 105; the pixel values ​​of the texels in decoded texture image 105 can be referred to as reconstructed values. It should be noted that the reconstructed value of any texel in decoded texture image 105 may be the same as or different from the pixel value of the corresponding pixel in texture image 104.

[0275] FIG4C is a schematic diagram of an exemplary decoding process of a texture image, wherein the decoding process of FIG4C corresponds to the encoding process of FIG2C .

[0276] 4A and 4C , illustratively, the decoding process of a texture image may be as follows: a first decoding module 401 may perform decoding 41 on any bitstream block 123 (one bitstream block 123 corresponds to one texture block) to obtain a coordinate quantized value 121 of a projection point corresponding to the pixel value. Next, a first inverse quantization module 402 may perform inverse quantization (or inverse quantization processing) 42 on the coordinate quantized value 121 of the projection point corresponding to the pixel value to obtain an inverse quantized coordinate value 127 of the projection point corresponding to the pixel value. Subsequently, an inverse transformation module 403 may perform inverse transformation 46 on the inverse quantized coordinate value 127 of the projection point corresponding to the pixel value to obtain a coordinate reconstructed value 129 of the projection point corresponding to the pixel value. Thereafter, an interpolation module 407 may perform interpolation 43 based on the coordinate reconstructed value 129 of the projection point corresponding to the pixel value to obtain a decoded texture block 125. After all code stream blocks 123 are decoded 41, inverse quantized 42, inverse transformed 46 and interpolated 43 in sequence, multiple decoded texture blocks 125 belonging to the same texture image can be obtained; then, the multiple decoded texture blocks 125 can be spliced ​​44 to obtain a decoded texture image 105.

[0277] It should be noted that the stitching 44 may be an optional step in the texture image decoding process.

[0278] FIG4D is a schematic diagram of an exemplary decoding process of a texture image, wherein the decoding process of FIG4D corresponds to the encoding process of FIG2D .

[0279] 4A and 4D , illustratively, the decoding process of a texture image may be as follows: a first decoding module 401 may perform decoding 41 on any code stream block 123 (one code stream block 123 corresponds to one region) to obtain a coordinate quantized value 121 of a projection point corresponding to the pixel value. Next, a first dequantization module 402 may perform dequantization (or dequantization processing) 42 on the coordinate quantized value 121 of the projection point corresponding to the pixel value to obtain a dequantized coordinate value 127 of the projection point corresponding to the pixel value. Subsequently, an inverse transformation module 403 may perform inverse transformation 46 on the dequantized coordinate value 127 of the projection point corresponding to the pixel value to obtain a coordinate reconstructed value 129 of the projection point corresponding to the pixel value. Thereafter, an interpolation module 407 may perform interpolation 43 based on the coordinate reconstructed value 129 of the projection point corresponding to the pixel value to obtain a decoded region 126. After decoding 41, inverse quantization 42, inverse transformation 46, and interpolation 43 are sequentially performed on the codestream blocks 123 corresponding to all regions belonging to the same texture block, multiple decoded regions 126 can be obtained. Subsequently, the multiple decoded regions 126 can be spliced ​​45 to obtain a decoded texture block 125. After all decoded texture blocks 125 belonging to the same texture image are obtained, the multiple decoded texture blocks 125 can be spliced ​​44 to obtain a decoded texture image 105.

[0280] It should be noted that splicing 44 and splicing 45 may be optional steps in the texture image encoding process.

[0281] For example, the second decoding module 404 can decode the encoded bitstream 104 to obtain quantized values ​​representing the endpoints of the fitted surface. Subsequently, the second dequantization module 405 can dequantize the quantized values ​​representing the endpoints of the fitted surface to obtain dequantized values ​​representing the endpoints of the fitted surface. Subsequently, the post-processing module 406 can perform post-processing based on the dequantized values ​​representing the endpoints of the fitted surface to obtain reconstructed values ​​representing the endpoints of the fitted surface. The interpolation module 407 can then perform interpolation 43 based on the reconstructed values ​​representing the endpoints of the fitted surface and the coordinate reconstructed values ​​129 of the projection points corresponding to the pixel values ​​to obtain the decoded texture image 105.

[0282] Fig. 5A is a schematic diagram illustrating an exemplary texture image decoding process 500. The decoding process 500 in Fig. 5A corresponds to the encoding process 300 in Fig. 3A.

[0283] S501: Obtain an encoded bitstream.

[0284] In one possible approach, during rendering, the terminal device or server may read the encoded code stream (such as the code stream of the texture image, or the code stream block of the texture block, or the code stream block of the region) generated by the encoding process 300 from the memory.

[0285] In one possible embodiment, during rendering, the terminal device may receive the encoded code stream (such as the code stream of the texture image, or the code stream block of the texture block, or the code stream block of the region) generated by the encoding process 300 sent by the server and store it in the memory, and then read the encoded code stream from the memory.

[0286] S502 , decoding the encoded code stream to obtain reconstruction values ​​of three or more endpoints and coordinate reconstruction values ​​of multiple projection points.

[0287] Exemplarily, the coordinate quantization values ​​of the plurality of projection points may be decoded from the encoded bitstream; then, the coordinate quantization values ​​of the plurality of projection points may be dequantized to obtain the coordinate reconstruction values ​​of the plurality of projection points.

[0288] It should be noted that the coordinate reconstruction value of each projection point obtained by decoding may be the same as or different from the coordinate of each projection point used for encoding, and this application does not impose any restrictions on this.

[0289] In one possible approach, quantized values ​​of three or more endpoints representing the geometric surface can be decoded from the encoded bitstream. Subsequently, the quantized values ​​of the three or more endpoints representing the geometric surface can be dequantized to obtain reconstructed values ​​of the three or more endpoints representing the geometric surface.

[0290] In one possible approach, quantized values ​​of three or more endpoints representing a geometric surface can be decoded from an encoded bitstream. Subsequently, the quantized values ​​of the three or more endpoints representing the geometric surface can be dequantized to obtain dequantized values ​​of the three or more endpoints representing the geometric surface. Thereafter, post-processing can be performed based on the dequantized values ​​of the three or more endpoints representing the geometric surface to obtain reconstructed values ​​of the three or more endpoints representing the geometric surface.

[0291] In one possible approach, the quantized values ​​of the first portion of endpoints and the quantized values ​​of the second portion of endpoints can be decoded from the bitstream. The quantized values ​​of the first portion of endpoints can then be dequantized to obtain dequantized values ​​of the first portion of endpoints. The quantized values ​​of the second portion of endpoints can then be dequantized to obtain reconstructed values ​​of the second portion of endpoints. Finally, post-processing can be performed based on the reconstructed values ​​of the second portion of endpoints and the dequantized values ​​of the first portion of endpoints to obtain reconstructed values ​​of the first portion of endpoints.

[0292] It should be noted that the reconstructed value of each endpoint obtained by decoding may be the same as or different from the value of each endpoint used for encoding, and this application does not impose any restrictions on this.

[0293] S503 : Generate reconstruction values ​​of a plurality of texture pixels in the texture image according to the reconstruction values ​​of three or more endpoints and the coordinate reconstruction values ​​of a plurality of projection points.

[0294] For example, the interpolation module 407 may perform calculations based on the coordinate reconstruction values ​​of the plurality of projection points and the reconstruction values ​​of three or more endpoints used to represent the geometric surface to generate the reconstruction values ​​of the plurality of texture pixels.

[0295] It should be noted that for different types of geometric surfaces, the interpolation module 407 performs calculations in different ways based on the coordinate reconstruction values ​​of multiple projection points and the reconstruction values ​​of three or more endpoints used to represent the geometric surface, which will be explained in detail later.

[0296] In one possible approach, pixel values ​​of all texture pixels in the texture image corresponding to the coordinate reconstruction values ​​of the projection points of the fitting points to the geometric surface are decoded from the encoded bitstream, and then S503 is executed to obtain the reconstruction values ​​of all texture pixels in the texture image, that is, the decoded texture image.

[0297] In one possible approach, the encoded bitstream includes multiple bitstream blocks. The coordinate reconstruction values ​​of the projection points of the fitting points onto the geometric surface corresponding to the pixel values ​​of all texture pixels contained in a texture block in the texture image are decoded from one bitstream block. Then, S503 is executed to obtain the reconstructed values ​​of all texture pixels in the texture block, i.e., the decoded texture block. S502 to S503 are then executed in a loop to obtain multiple decoded texture blocks. Thereafter, the multiple decoded texture blocks can be spliced ​​together to obtain a decoded texture image. It should be understood that the decoded texture blocks can be used directly without splicing them together.

[0298] In one possible approach, the encoded code stream includes multiple code stream blocks, and the coordinate reconstruction values ​​of the projection points of the fitting points to the geometric surface corresponding to the pixel values ​​of all texture pixels contained in an area of ​​the texture image are decoded from one code stream block, and then S503 is executed to obtain the reconstructed values ​​of all texture pixels in the area, that is, the decoded area. S502 to S503 are subsequently executed in a loop to obtain multiple decoded areas. Afterwards, multiple decoded areas can be spliced ​​to obtain decoded texture blocks; and then multiple decoded texture blocks can be spliced ​​to obtain a decoded texture image. It should be understood that it is also possible to directly use the decoded areas without splicing them. Alternatively, it is also possible to directly use the decoded texture blocks without splicing them.

[0299] The following describes the decoding process of the texture image in conjunction with the rendering process.

[0300] FIG5B is a schematic diagram illustrating an exemplary rendering process and a decoding process of a texture image.

[0301] 5B , illustratively, the renderer 114 may include a geometry processing unit 1141 , a rasterization unit 1142 , a pixel processing unit 1143 , and a texture sampling unit 1144 .

[0302] For example, the geometry processing unit 1141, the rasterization unit 1142, the pixel processing unit 1143, and the texture sampling unit 1144 may be implemented by a GPU; the geometry processing unit 1141, the rasterization unit 1142, the pixel processing unit 1143, and the texture sampling unit 1144 may be hardware modules. It should be understood that the renderer 114 of the present application may include more or fewer units than those shown in FIG. 5B , and the present application does not limit this.

[0303] 5B , illustratively, the decoder 113 may include a number determination module 1131, a loading module 1132, and a decoding module 1133. It should be understood that the decoder 113 of the present application may include more or fewer units than those shown in FIG. 5B , and the present application does not limit this. The decoding module 1133 may include the modules shown in FIG. 4A , or may include more or fewer modules than those shown in FIG. 4A , and the present application does not limit this.

[0304] For example, the number determination module 1131, the loading module 1132, and the decoding module 1133 can be implemented by a GPU; the number determination module 1131, the loading module 1132, and the decoding module 1133 can be hardware modules. It should be understood that when the decoder 113 is independent of the GPU, the modules included in the decoder 113 can be software modules (also called logic modules), and this application does not limit this.

[0305] 5B , illustratively, during rendering, the mesh 103 and the code stream block 123 of texture data may be loaded into the memory 111 ; then, the mesh 103 may be loaded from the memory 111 into the geometry processing unit 1141 , which performs geometric processing (such as coordinate transformation, determining the color of vertex coordinates (optional), calculating lighting (optional) etc.) on the mesh 103 to obtain the processed mesh and output it to the rasterization unit 1142 .

[0306] For example, if the geometry processing unit 1141 determines the color of a vertex coordinate, it can calculate the pixel coordinates of the required texture pixels and send them to the texture sampling unit 1144. The texture sampling unit 1144 can send the pixel coordinates of the texture pixels to the number determination module 1131 in the decoder 113. The number determination module 1131 calculates the number of the codestream block corresponding to the pixel coordinates based on the pixel coordinates of the texture pixels, the texture block size, and the codestream block size, and outputs the number to the loading module 1132. The loading module 1132 can then load the codestream block corresponding to the codestream block number received from the number determination module 1131 from the memory 111 and send the codestream block to the decoding module 1133. The decoding module 1133 performs steps 41, 42, and 43 in FIG. 4B to obtain a decoded texture block. The decoded texture block can then be output to the texture sampling unit 1144, which in turn outputs the decoded texture block to the geometry processing unit 1141. Then, the geometry processing unit 1141 may perform geometry processing according to the decoded texture block.

[0307] Exemplarily, the rasterization unit 1142 can rasterize the processed mesh (for example, map the vertices on the mesh to the screen, obtain 2D pixels, and determine the pixels that need to be shaded), obtain rasterization information (such as 2D pixels and identification of pixel points that need to be shaded) and output it to the pixel processing unit 1143.

[0308] Exemplarily, the pixel processing unit 1143 may perform pixel processing (such as calculating lighting, calculating shadows, and texture mapping) and output a rendered image.

[0309] For example, when the pixel processing unit 1143 needs to use a texture image during pixel processing (e.g., texture mapping), it can calculate the pixel coordinates of the required texture pixels and send them to the texture sampling unit 1144. The texture sampling unit 1144 can then obtain the decoded texture block in the manner described above and return it to the pixel processing unit 1143. This description will not be repeated here.

[0310] Since the surface properties of spatial objects (such as color, normal, etc.) are mostly continuous; therefore, the texture image has the characteristics of block-like planar distribution, and these characteristics can constitute a continuous surface in space. Furthermore, compared with the prior art that fits the pixel values ​​of texture pixels in the texture image to a straight line, the present application fits the pixel values ​​of texture pixels in the texture image to a geometric surface, which can reduce the fitting error. In other words, compared with the prior art, the present application can reduce the error between the pixel value of the texture pixel and the coordinates of the corresponding projection point; in this way, the error between the reconstructed value of the texture pixel reconstructed by the present application based on the encoded data (that is, the coordinates of the encoded projection point) and the pixel value of the texture pixel (that is, the original value) is smaller (that is, the encoding error is smaller). When the bit rate of the encoded texture image is fixed (that is, at the same bit rate), the reconstructed value of the texture pixel reconstructed by the present application is more accurate compared with the prior art; thus, the present application can improve the reconstruction quality of the texture image and the quality of the rendered image.

[0311] The following takes encoding and decoding a texture block as an example to illustrate the process of encoding the values ​​of three or more endpoints used to represent a geometric surface, determining the coordinates of the projection points of the fitting points corresponding to the pixel values ​​of multiple texture pixels in the texture block to the geometric surface, and fitting the pixel values ​​of multiple texture pixels in the texture block to obtain the geometric surface.

[0312] Exemplarily, the three or more endpoints include: a first endpoint and a second endpoint; encoding the value of the first endpoint and the value of the second endpoint includes: determining the residual between the value of the second endpoint and the value of the first endpoint as the residual value of the second endpoint; wherein the value of the first endpoint is used as the predicted value of the second endpoint; encoding the residual value of the second endpoint and the value of the first endpoint.

[0313] In this way, when the residual value of the second endpoint is small, the bits required to encode the residual value of the second endpoint are fewer than the bits required to encode the value of the second endpoint. When the bit rate of the encoded texture image is fixed, the bits used to encode the value of the first endpoint can be increased. This can improve the accuracy of the reconstructed value of the first endpoint obtained by the decoder, as well as the accuracy of the reconstructed value of the second endpoint determined based on the reconstructed value of the first endpoint and the residual value of the second endpoint. This can improve the quality of the texture image reconstruction and the quality of the rendered image. When the bit rate of the encoded texture image is not fixed, the bit rate can be reduced.

[0314] Exemplarily, the three or more endpoints also include: other endpoints; encoding the values ​​of other endpoints, including: determining the residual between the values ​​of the other endpoints and the value of the first endpoint as the residual value of the other endpoints; wherein the value of the first endpoint is used as the predicted value of the other endpoints; encoding the residual value of the other endpoints.

[0315] Similarly, when the residual values ​​of other endpoints are small, fewer bits are required to encode the residual values ​​of the other endpoints compared to the bits required to encode the values ​​of the other endpoints. When the bit rate of the encoded texture image is fixed, the bits used to encode the value of the first endpoint can be increased. This improves the accuracy of the reconstructed value of the first endpoint obtained by the decoder, as well as the accuracy of the reconstructed values ​​of the other endpoints determined based on the reconstructed value of the first endpoint and the residual values ​​of the other endpoints. When the bit rate of the encoded texture image is not fixed, the bit rate can be reduced.

[0316] Exemplarily, when there are multiple other endpoints, the residual value of each other endpoint can be encoded.

[0317] In one possible approach, the values ​​of other endpoints can also be encoded directly.

[0318] In one possible approach, when there are two other endpoints (including a third endpoint and a fourth endpoint), the residual between the value of the third endpoint and the value of the fourth endpoint (ie, the residual value of the fourth endpoint) can also be encoded, as well as the value of the third endpoint.

[0319] Similarly, when there are multiple other endpoints, the residual between the values ​​of any two endpoints and the value of any one of the two endpoints can be encoded; this application does not impose any restrictions on this.

[0320] Exemplarily, decoding the encoded code stream to obtain a reconstructed value of a first endpoint and a reconstructed value of a second endpoint includes: parsing a first identifier from the encoded code stream; wherein the first identifier indicates whether to encode a residual value of the second endpoint, and the residual value of the second endpoint is the residual between the value of the second endpoint and the value of the first endpoint; when the value of the first identifier is a preset identifier value, decoding the encoded code stream to obtain a reconstructed value of the first endpoint and a reconstructed value of the residual value of the second endpoint; adding the reconstructed value of the first endpoint to the reconstructed value of the residual value of the second endpoint to obtain a reconstructed value of the second endpoint; wherein the reconstructed value of the first endpoint is used as a predicted value of the second endpoint.

[0321] Exemplarily, decoding the encoded code stream to obtain reconstructed values ​​of other endpoints includes: when the value of the first identifier is a preset identifier value, decoding the encoded code stream to obtain reconstructed values ​​of residual values ​​of other endpoints; wherein the residual values ​​of other endpoints are the residuals between the values ​​of the other endpoints and the value of the first endpoint; adding the reconstructed value of the first endpoint to the reconstructed values ​​of the residual values ​​of the other endpoints to obtain the reconstructed values ​​of the other endpoints.

[0322] The following explanation is given using an example where three endpoints are used to represent a geometric surface.

[0323] Fig. 6A is a schematic diagram illustrating an exemplary texture image encoding process 600. Fig. 6A shows a method of performing fitting processing on a plurality of texture pixels in a texture block to obtain a geometric surface.

[0324] S601 , performing fitting processing on pixel values ​​of a plurality of texture pixels to obtain a plurality of candidate geometric surfaces, where the plurality of candidate geometric surfaces correspond to one type.

[0325] Exemplarily, one type of geometric surface is used to fit the pixel values ​​of multiple texture pixels in the texture block, and multiple candidate geometric surfaces of the same type can be obtained.

[0326] For example, a PCA method may be used to perform fitting processing on the pixel values ​​of multiple texture pixels in a texture block to obtain multiple candidate planes.

[0327] For example, a least squares method may be used to fit the pixel values ​​of multiple texture pixels in a texture block to obtain multiple candidate curved surfaces. The candidate curved surfaces may be any of a parabola, a hyperbola, and the like (it should be understood that multiple candidate curved surfaces may all be parabolas, or multiple candidate curved surfaces may all be hyperbolas, and so on), and this application does not impose any limitation on this.

[0328] For example, a gradient descent method may be used to fit the pixel values ​​of multiple texture pixels in a texture block to obtain multiple candidate geometric bodies. The candidate geometric bodies may be ellipsoids, spheres, etc. (it should be understood that multiple candidate geometric bodies may all be ellipsoids, or multiple candidate geometric bodies may all be spheres, etc.), and this application does not impose any limitation on this.

[0329] S602: Determine one of the N candidate geometric surfaces with the smallest corresponding fitting error as the geometric surface.

[0330] Wherein, N is a positive integer, N can be set as required, and this application does not impose any restrictions on this.

[0331] For example, the fitting error of each candidate geometric surface can be calculated. Specifically, for a current candidate geometric surface among multiple candidate geometric surfaces, the sum of the distances from the fitting point corresponding to the pixel values ​​of all texture pixels in the texture block to the current candidate geometric surface can be calculated to obtain the fitting error of the current candidate geometric surface. The current candidate geometric surface can be any one of the multiple candidate geometric surfaces.

[0332] Next, N candidate geometric surfaces with the smallest fitting errors are determined; and one candidate geometric surface is selected as the geometric surface from the N candidate geometric surfaces with the smallest corresponding fitting errors.

[0333] Optionally, N=1, that is, the candidate geometric surface with the smallest corresponding fitting error is used as the geometric surface.

[0334] S603, obtaining coordinates of a plurality of projection points in the geometric surface; wherein the plurality of projection points correspond one-to-one to pixel values ​​of a plurality of texture pixels, and the coordinates of the projection points include two or more dimensions.

[0335] For example, S603 may refer to the description of 302 above, which will not be repeated here.

[0336] S604: Obtain the values ​​of the three endpoints used to represent the geometric surface.

[0337] In one possible approach, when the geometric surface is a plane, three endpoints can be selected from the plane to form a vertical coordinate system so that the coordinates of all projection points are within the range [0, 1], and used as the three endpoints for representing the plane. For example, the endpoints ept0, ept1, and ept2 in Figure 3B can be used as the three endpoints for representing the plane.

[0338] In one possible approach, when the geometric surface is a plane, three endpoints that can form a non-vertical coordinate system and enable the coordinates of all projection points to be within the range of [0,1] can be taken from the plane as the three endpoints used to represent the plane.

[0339] It should be understood that when selecting endpoints in a plane, three endpoints can also be taken from the plane that can form a vertical coordinate system (or non-vertical coordinate system) and can make the coordinates of all projection points within other ranges that are easy to encode, as the three endpoints used to represent the plane; this application does not impose any restrictions on this.

[0340] For example, when the geometric surface is an elliptical paraboloid, the three endpoints selected to represent the elliptical paraboloid may be: the vertex of the elliptical paraboloid, the direction of the axis of symmetry, and the coefficient of the elliptical equation. It should be understood that the three endpoints used to represent different curved surfaces may be different.

[0341] For example, when the geometric surface is an ellipsoid, the four endpoints selected to represent the ellipsoid may be: the center of the ellipsoid, the directions of the two symmetry axes, and the coefficients of the ellipse equation. It should be understood that the three or more endpoints used to represent different geometric bodies may be different.

[0342] For example, the method of encoding the coordinates of the multiple projection points may refer to the description of S605.

[0343] S605 , using a coding algorithm combination with the smallest coding error to encode the coordinates of the multiple projection points.

[0344] Exemplarily, M (M is a positive integer) coding mode combinations can be pre-constructed, and each coding mode combination can include one or more coding modes. Exemplarily, the coding mode can be used to describe the preprocessing method and the coding algorithm. Optionally, the coding mode can also be used to describe the storage format, etc. Among them, the preprocessing method includes a residual processing method or a non-residual processing method. The coding algorithm refers to a method for reducing the redundancy of the data to be encoded (including the values ​​of three or more endpoints used to represent the geometric surface and the coordinates of multiple projection points), such as the above-mentioned entropy coding, ASCII coding, etc. The storage format refers to the format of the encoded data obtained by encoding (for example, ASCII code, BCD code, Gray code, etc.), or the format of the encoded data in the encoded code stream.

[0345] That is to say, each coding mode combination may include one or more coding modes. It can be understood that each coding mode combination may include a preprocessing method combination (including one or more preprocessing methods), a coding algorithm combination (including one or more coding algorithms) and a storage format combination (including one or more storage formats).

[0346] In one possible approach, for encoding the coordinates of multiple projection points, the preprocessing method corresponding to each encoding mode in the optional encoding mode combination is a non-residual processing method.

[0347] Exemplarily, when a coding mode combination includes one coding mode, the coding mode combination can be used to encode all dimensions of the coordinates of each projection point. When a coding mode combination includes multiple coding modes, one coding mode in the coding mode combination can be used to encode one dimension or multiple dimensions of the coordinates of each projection point.

[0348] Similarly, when the coding algorithm combination of a certain coding mode combination includes one coding algorithm, the coding algorithm can be used to encode all dimensions of the coordinates of each projection point. When the coding algorithm combination of a certain coding mode combination includes multiple coding algorithms, one coding algorithm of the coding algorithm combination can be used to encode one or more dimensions of the coordinates of each projection point.

[0349] FIG6B is a schematic diagram showing an exemplary process of selecting a coding mode combination with the smallest coding error. Specifically, the process of selecting a coding mode combination with the smallest coding error can refer to the following S1 to S8:

[0350] S1, using the coding mode in the i-th coding mode combination to encode the coordinates of multiple projection points, where the initial value of i is 1.

[0351] Exemplarily, when the i-th coding mode combination includes a coding mode, the coding mode can be used to encode the entirety of the coordinates of two or more dimensions of each projection point. For example, the coordinates of the projection point A1 of the fitting point A to the geometric surface are (u1, v1), and the entirety of u1 and v1 can be encoded according to a coding mode included in the i-th coding mode combination. For example, after concatenating u1 and v1, the concatenated data is encoded according to a coding mode included in the i-th coding mode combination.

[0352] Exemplarily, when the i-th coding mode combination includes a coding mode, the coding mode can be used to independently encode the coordinates of two or more dimensions of each projection point. For example, the coordinates of the projection point A1 of the fitting point A to the geometric surface are (u1, v1). According to the i-th coding mode combination including a coding mode, u1 and v1 can be independently encoded.

[0353] Exemplarily, when the i-th coding mode combination includes multiple coding modes (the number of coding modes is the same as the dimensions included in the coordinates of the projection points), one coding mode in the i-th coding mode combination is used to encode one dimension of the coordinates of each projection point. For example, if the coordinates of the projection point A1 of the fitting point A onto the geometric surface are (u1, v1), u1 can be encoded according to one coding mode included in the i-th coding mode combination, and v1 can be encoded according to another coding mode included in the i-th coding mode combination.

[0354] Exemplarily, when the i-th coding mode combination includes multiple coding modes (the number of coding modes is greater than 1 and less than the dimensions included in the coordinates of the projection point), one coding mode in the i-th coding mode combination is used to encode at least one dimension of the coordinates of each projection point. For example, the coordinates of the projection point A1 of the fitting point A onto the geometric surface are (u1, v1, r1). u1 and v1 can be encoded according to one coding mode included in the i-th coding mode combination; and r1 can be encoded according to another coding mode included in the i-th coding mode combination.

[0355] S2: Determine whether the residual between the value of the first endpoint and the value of the second endpoint is within a preset range.

[0356] Exemplarily, the three or more endpoints used to represent the geometric surface include: a first endpoint, a second endpoint, and a third endpoint.

[0357] Exemplarily, when the residual between the value of the first endpoint and the value of the second endpoint (also referred to as the residual value of the second endpoint) is within a preset range, the value of the first endpoint, the residual between the value of the first endpoint and the value of the second endpoint, and the value of the third endpoint can be encoded; that is, S3 is executed. In this way, when the residual value of the second endpoint is small, the bits required to encode the residual value of the second endpoint are fewer than the bits required to encode the value of the second endpoint; when the bit rate of the encoded texture image is fixed, the bits used to encode the value of the first endpoint can be increased. In this way, the accuracy of the reconstructed value of the first endpoint obtained by decoding at the decoding end and the accuracy of the reconstructed value of the second endpoint determined based on the reconstructed value of the first endpoint and the residual value of the second endpoint can be improved; thereby, the reconstruction quality of the texture image and the quality of the rendered image can be improved. When the bit rate of the encoded texture image is not fixed, the bit rate can be reduced.

[0358] For example, the preset range can be set as required, such as [-64, 63], and this application does not impose any limitation on this.

[0359] For example, when the residual between the first endpoint and the second endpoint is not within a preset range, the values ​​of the first, second, and third endpoints can be directly encoded. This is because when the residual between the first endpoint and the second endpoint is not within the preset range, that is, the difference between the first and second endpoints is significant; in this case, the number of bits required to encode the residual between the first and second endpoints is equal to the number of bits required to encode the second endpoint. This ensures that the number of bits used to encode the first endpoint remains constant, and the accuracy of the reconstructed value of the first endpoint obtained by the decoder is not improved. Because the reconstructed value of the first endpoint contains errors, and the residual of the second endpoint also contains errors, the reconstructed value of the second endpoint, obtained based on the reconstructed value of the first endpoint and the residual, will have a greater error than the error of the reconstructed value of the second endpoint directly decoded from the encoded bitstream, thereby reducing the accuracy of the reconstructed value of the second endpoint. Therefore, to ensure the accuracy of the reconstructed value of the second endpoint, the values ​​of the first and second endpoints can be directly encoded.

[0360] S3, encoding the residual between the value of the first endpoint and the value of the second endpoint, the value of the first endpoint, and the value of the third endpoint.

[0361] Exemplarily, the data input to the encoder in S3 is: a residual between the value of the first endpoint and the value of the second endpoint, the value of the first endpoint, and the value of the third endpoint.

[0362] For example, the number of bits used to encode the values ​​representing the three endpoints of the geometric surface can be determined based on the total number of bits in the codestream block and the number of bits used to encode the coordinates of multiple projection points. Subsequently, the quantization parameter for the residual difference between the value of the first endpoint and the value of the second endpoint, the size of the value of the first endpoint, the size of the value of the third endpoint, and the number of bits used to encode the values ​​representing the three endpoints of the geometric surface can be determined based on the size of the residual difference between the value of the first endpoint and the value of the second endpoint, the size of the value of the first endpoint, the size of the value of the third endpoint, and the number of bits used to encode the values ​​representing the three endpoints of the geometric surface.

[0363] Then, the residual between the value of the first endpoint and the value of the second endpoint may be quantized and then encoded based on the quantization parameter of the residual between the value of the first endpoint and the value of the second endpoint. The value of the first endpoint may be quantized and then encoded based on the quantization parameter of the value of the first endpoint. And the value of the third endpoint may be quantized and then encoded based on the quantization parameter of the value of the third endpoint.

[0364] S4, respectively encode the value of the first endpoint, the value of the second endpoint, and the value of the third endpoint.

[0365] Exemplarily, the data input to the encoder in S4 are: the value of the first endpoint, the value of the second endpoint, and the value of the third endpoint.

[0366] For example, the number of bits used to encode the values ​​representing the three endpoints of the geometric surface can be determined based on the total number of bits in the codestream block and the number of bits used to encode the coordinates of multiple projection points. Subsequently, the quantization parameter for the value of the first endpoint, the quantization parameter for the value of the second endpoint, and the quantization parameter for the value of the third endpoint can be determined based on the magnitude of the value of the first endpoint, the magnitude of the value of the second endpoint, and the magnitude of the value of the third endpoint, and the number of bits used to encode the values ​​representing the three endpoints of the geometric surface.

[0367] Then, the value of the first endpoint may be quantized and then encoded according to the quantization parameter of the value of the first endpoint. The value of the second endpoint may be quantized and then encoded according to the quantization parameter of the value of the second endpoint. And the value of the third endpoint may be quantized and then encoded according to the quantization parameter of the value of the third endpoint.

[0368] It should be noted that the value of the first endpoint, the value of the second endpoint, the value of the third endpoint, and the residual between the value of the first endpoint and the value of the second endpoint can use the same encoding algorithm or different encoding algorithms, and this application does not impose any restrictions on this. The specific implementation method can be similar to encoding two or more dimensions of the coordinates of each projection point (that is, the value of the first endpoint, the value of the second endpoint, the value of the third endpoint, and the residual between the value of the first endpoint and the value of the second endpoint can each be regarded as a dimension), which will not be repeated here.

[0369] S5, determining the coding error corresponding to the i-th coding mode combination.

[0370] Subsequently, decoding can be performed based on the encoded data obtained by S1 encoding to obtain coordinate reconstruction values ​​of multiple projection points. Furthermore, decoding can be performed based on the encoded data obtained by S3 or S4 encoding to obtain reconstruction values ​​of three or more endpoints representing the geometric surface. Subsequently, based on the reconstruction values ​​of the three or more endpoints representing the geometric surface and the coordinate reconstruction values ​​of the multiple projection points, reconstruction values ​​of multiple texture pixels in the texture image are determined. Next, based on the pixel values ​​and reconstruction values ​​of the texture pixels, the coding error corresponding to the i-th coding mode combination is determined.

[0371] Exemplarily, the sum of square differences between pixel values ​​and reconstructed values ​​of a plurality of texture pixels may be calculated to obtain the coding error corresponding to the i-th coding mode combination.

[0372] S6, determine whether i is equal to M.

[0373] When i is not equal to M, execute S7; when i is equal to M, execute S8.

[0374] S7, increase i by 1.

[0375] After executing S7, you can return to execute S1.

[0376] S8, selecting the coding mode combination with the smallest coding error.

[0377] Exemplarily, when the coding mode combination with the minimum coding error includes a coding mode, the coding mode can be used to encode the entirety of the coordinates of two or more dimensions of each projection point. For example, the coordinates of the projection point A1 of the fitting point A to the geometric surface are (u1, v1), and the entirety of u1 and v1 can be encoded according to a coding mode included in the coding mode combination with the minimum coding error. For example, after concatenating u1 and v1, the concatenated data is encoded according to a coding mode included in the coding mode combination with the minimum coding error.

[0378] Exemplarily, when the coding mode combination with the minimum coding error includes a coding mode, the coding mode can be used to independently encode the coordinates of two or more dimensions of each projection point. For example, if the coordinates of the projection point A1 of the fitting point A to the geometric surface are (u1, v1), the coding mode combination with the minimum coding error can include a coding mode to independently encode u1 and v1.

[0379] Exemplarily, when the coding mode combination with the minimum coding error includes multiple coding modes (the number of coding modes is the same as the dimensions included in the coordinates of the projection points), one coding mode in the coding mode combination with the minimum coding error is used to encode one dimension of the coordinates of each projection point. For example, if the coordinates of the projection point A1 of the fitting point A onto the geometric surface are (u1, v1), u1 can be encoded according to one coding mode included in the coding mode combination with the minimum coding error, and v1 can be encoded according to another coding mode included in the coding mode combination with the minimum coding error.

[0380] Exemplarily, when the coding mode combination with the minimum coding error includes multiple coding modes (the number of coding modes is greater than 1 and less than the dimensions included in the coordinates of the projection point), one coding mode in the coding mode combination with the minimum coding error is used to encode at least one dimension of the coordinates of each projection point. For example, if the coordinates of the projection point A1 of the fitting point A to the geometric surface are (u1, v1, r1), u1 and v1 can be encoded according to one coding mode included in the coding mode combination with the minimum coding error; and r1 can be encoded according to another coding mode included in the coding mode combination with the minimum coding error.

[0381] For example, the process of encoding the three endpoints for representing the geometric surface may refer to the following description of S606 to S609:

[0382] S606: Determine the residual between the value of the second endpoint and the value of the first endpoint as the residual value of the second endpoint; wherein the value of the first endpoint is used as the predicted value of the second endpoint.

[0383] S607: Determine whether the residual value of the second endpoint is within a preset range.

[0384] Illustratively, when the residual value of the second endpoint is within a preset range, S608 may be executed; otherwise, S609 may be executed.

[0385] S608: Encode the residual value of the second endpoint, the value of the first endpoint, and the value of the third endpoint.

[0386] Exemplarily, any encoding algorithm may be used to encode the residual between the value of the first endpoint and the value of the second endpoint, the value of the first endpoint, and the value of the third endpoint; this application does not impose any limitation on this.

[0387] S609 , respectively encode the value of the first endpoint, the value of the second endpoint, and the value of the third endpoint.

[0388] Illustratively, any encoding algorithm may be used to encode the value of the first endpoint, the value of the second endpoint, and the value of the third endpoint, respectively.

[0389] For example, S606 to S609 may refer to the description of S2 to S5 above, which will not be repeated here.

[0390] In this way, by selecting a candidate geometric surface with a smaller fitting error as the final geometric surface, the fitting error can be reduced. Thus, the reconstructed value obtained by the present application is more accurate, and thus, at the same bit rate, the pixel value of the texture pixel (ie, the reconstructed value) is more accurate.

[0391] 7A to 7C are schematic diagrams illustrating exemplary encoded code stream structures.

[0392] 7A(1), exemplarily, the encoded code stream obtained according to the encoding algorithm of process 600 may include encoded data for representing the values ​​of three or more endpoints of a geometric surface: encoded data for the value of a first endpoint, encoded data for the residual value of a second endpoint, ..., encoded data for the value of a Pth endpoint; and encoded data for the coordinates of a plurality of projection points: encoded data for the coordinates of projection point 1, ..., encoded data for the coordinates of projection point R.

[0393] Wherein, P is an integer greater than or equal to 3, and R is a positive integer.

[0394] 7A(2), exemplarily, the encoded code stream obtained according to the encoding algorithm of process 600 may include encoded data for representing the values ​​of three or more endpoints of a geometric surface: encoded data for the value of a first endpoint, encoded data for the value of a second endpoint, ..., encoded data for the value of a P-th endpoint; and encoded data for the coordinates of a plurality of projection points: encoded data for the coordinates of projection point 1, ..., encoded data for the coordinates of projection point R.

[0395] It should be understood that the present application does not limit the order of the encoded data used to represent the values ​​of three or more endpoints of a geometric surface and the encoded data of the coordinates of multiple projection points in the encoded code stream.

[0396] For example, a first flag and a second flag may be added to the encoded bitstream, as shown in FIG7B . The first flag indicates whether to encode the residual between the value of the first endpoint and the value of the second endpoint, and the second flag indicates the encoding algorithm corresponding to the coordinates of each projection point in the multiple projection points. This ensures that the decoder can reconstruct the pixel value of the texture pixel.

[0397] It should be understood that the present application is not limited to the encoded data used to represent the values ​​of three or more endpoints of a geometric surface, the encoded data of the coordinates of multiple projection points, and the order of the first identifier and the second identifier in the encoded code stream.

[0398] Exemplarily, the first identifier may also indicate an encoding algorithm for representing the values ​​of three or more endpoints of a geometric surface.

[0399] Exemplarily, the first identifier may also indicate a storage format of encoded data representing the values ​​of three or more endpoints of a geometric surface.

[0400] For example, the first identifier is represented by 6 bits, the first bit indicates whether residual processing is performed on the three or more endpoints used to represent the geometric surface; the second to fourth bits indicate the encoding algorithm used to represent the values ​​of the three or more endpoints of the geometric surface; the fifth to sixth bits indicate the storage format of the encoded data used to represent the values ​​of the three or more endpoints of the geometric surface.

[0401] For example, if process 600 executes S608, and S608 adopts an entropy coding algorithm, and the storage format of the encoded data for representing the values ​​of three or more endpoints of a geometric surface is a BCD format, then the first identifier is 000001. The first bit "0" in "000001" indicates the residual between the encoded value of the first endpoint and the value of the second endpoint, the second to fourth bits "000" indicate that the encoding algorithm for representing the values ​​of three or more endpoints of a geometric surface is an entropy coding algorithm, and the fifth to sixth bits "01" indicate that the storage format of the encoded data for representing the values ​​of three or more endpoints of a geometric surface is a BCD format.

[0402] For example, if process 600 executes S609, and S609 adopts an entropy coding algorithm, and the storage format of the encoded data for representing the values ​​of three or more endpoints of a geometric surface is a BCD format, then the first identifier is 100001. The first bit "1" in "100001" indicates that the residual between the value of the first endpoint and the value of the second endpoint is not encoded, the second to fourth bits "000" indicate that the encoding algorithm for representing the values ​​of three or more endpoints of a geometric surface is an entropy coding algorithm, and the fifth to sixth bits "01" indicate that the storage format of the encoded data for representing the values ​​of three or more endpoints of a geometric surface is a BCD format.

[0403] For example, if process 600 executes S608, and S608 adopts the ASCII encoding algorithm, and the storage format of the encoded data for representing the values ​​of three or more endpoints of the geometric surface is the ASCII format, then the first identifier is 000110. The first bit "0" in "000110" indicates the residual between the encoded value of the first endpoint and the value of the second endpoint, the second to fourth bits "001" indicate that the encoding algorithm for representing the values ​​of three or more endpoints of the geometric surface is the ASCII encoding algorithm, and the fifth to sixth bits "10" indicate that the storage format of the encoded data for representing the values ​​of three or more endpoints of the geometric surface is the ASCII format.

[0404] Exemplarily, when the codec does not pre-agreed on the reference endpoint of the current endpoint, the first identifier may also indicate that the current endpoint corresponds to the reference endpoint. For example, when the residual value of the second endpoint and the value of the first endpoint are encoded, the first identifier may include the identifier of the first endpoint.

[0405] Exemplarily, the second identifier may further indicate whether residual processing is performed on the coordinates of the multiple projection points.

[0406] Exemplarily, the second identifier may also indicate a storage format of the encoded data of the coordinates of the plurality of projection points.

[0407] For example, the second identifier is represented by 6 bits, the first bit indicates whether residual processing is performed on the coordinates of the multiple projection points; the second to fourth bits indicate the encoding algorithm of all dimensions of the coordinates of the multiple projection points; the fifth to sixth bits indicate the storage format of the encoded data of the coordinates of the multiple projection points. In process 600, S605 does not perform residual processing on the coordinates of the multiple projection points; if S604 uses an entropy coding algorithm and the storage format of the encoded data of the coordinates of the multiple projection points is BCD format, the second identifier is 100001. If S605 uses an ASCII encoding algorithm and the storage format of the encoded data of the coordinates of the multiple projection points is ASCII format, the second identifier is 100110.

[0408] For example, the second identifier is represented by 11 bits, the first bit indicates whether residual processing is performed on the coordinates of multiple projection points; the second to fourth bits indicate the encoding algorithm of the first dimension of the coordinates of each projection point, and the fifth to seventh bits indicate the encoding algorithm of the second dimension of the coordinates of each projection point; the eighth to ninth bits indicate the storage format of the encoded data of the first dimension of the coordinates of each projection point; and the tenth to eleventh bits indicate the storage format of the encoded data of the second dimension of the coordinates of each projection point. In process 600, S605 does not perform residual processing on the coordinates of multiple projection points; if S605 uses an entropy encoding algorithm for the first dimension of the coordinates of each projection point, and the storage format of the encoded data of the first dimension of the coordinates of each projection point is BCD format; and S605 uses an ASCII encoding algorithm for the second dimension of the coordinates of each projection point, and the storage format of the encoded data of the second dimension of the coordinates of each projection point is ASCII format; then the second identifier is 10000100110.

[0409] In one possible approach, a third identifier may also be carried in the encoded bitstream, where the third identifier indicates the type of the geometric surface, as shown in FIG7C . For example, the third identifier may be represented by three bits, such as 000 indicating that the type of the geometric surface is a plane, 001 indicating that the type of the geometric surface is a parabola, 010 indicating that the type of the geometric surface is a hyperboloid, 011 indicating that the type of the geometric surface is an ellipsoid, and 100 indicating that the type of the geometric surface is a hyperboloid. It should be understood that this application does not limit the number of bits of the third identifier.

[0410] It should be understood that the present application is not limited to the encoded data used to represent the values ​​of three or more endpoints of a geometric surface, the encoded data of the coordinates of multiple projection points, and the order of the first identifier, the second identifier, and the third identifier in the encoded code stream.

[0411] It should be understood that the codec can pre-agreed on the encoding algorithm used to represent the values ​​of three or more endpoints of a geometric surface, whether to perform residual processing on the values ​​of three or more endpoints of a geometric surface, the storage format of the encoded data used to represent the values ​​of three or more endpoints of a geometric surface in the encoded bitstream, and the reference endpoint of the current endpoint. In this way, there is no need to carry the first identifier in the encoded bitstream.

[0412] It should be understood that the codec can pre-agreed on the encoding algorithm for the coordinates of the multiple projection points, whether the coordinates of the multiple projection points are subjected to residual processing, and the storage format of the encoded data of the coordinates of the multiple projection points. In this way, there is no need to carry the second identifier in the encoded bitstream.

[0413] It should be understood that the codec can pre-agreed on the type of geometric surface; in this way, there is no need to carry the third identifier in the encoded bitstream.

[0414] That is, the first, second, and third identifiers are optional in the encoded bitstream. When the encoded bitstream does not carry at least one of the first, second, and third identifiers, the number of bits used to encode the coordinates of multiple projection points or the number of bits used to encode the values ​​of three or more endpoints representing a geometric surface can be increased; thereby, the encoding accuracy of the coordinates of the multiple projection points or the encoding accuracy of the values ​​of three or more endpoints representing a geometric surface can be increased.

[0415] Exemplarily, the encoding end may include a device for transmitting an encoded code stream; the device for transmitting the encoded code stream may include: a transmitter and at least one storage medium, the at least one storage medium being used to store the encoded code stream generated by the above embodiment; the transmitter being used to obtain the encoded code stream from the storage medium and send the encoded code stream to the end-side device via a transmission medium.

[0416] Exemplarily, after generating an encoded bitstream, the encoding end may send the encoded bitstream to a system for distributing the encoded bitstream. The system for distributing the encoded bitstream may include: at least one storage medium and a streaming device; the storage medium is configured to store the at least one encoded bitstream generated in the above embodiment; and the streaming device is configured to obtain a target encoded bitstream from the at least one storage medium and send the target encoded bitstream to the end-side device, wherein the streaming device includes a content server or a content distribution server.

[0417] FIG8 is a schematic diagram illustrating an exemplary texture image decoding process 800 , wherein the decoding process 800 corresponds to the encoding process 600 described above.

[0418] S801: Obtain an encoded bitstream.

[0419] Exemplarily, the decoding end may include a device for storing the encoded code stream. The device for storing the encoded code stream may include: a receiver for receiving the encoded code stream; and at least one storage medium for storing the encoded code stream. The encoded code stream may then be decoded with reference to S802 to S809.

[0420] S802: parse the encoded code stream to obtain a first identifier, a second identifier, and a third identifier.

[0421] Exemplarily, the decoder may parse the encoded code stream to obtain the first identifier, the second identifier, and the third identifier.

[0422] Afterwards, on one hand, the encoded code stream can be decoded according to the encoding mode corresponding to the first identifier to obtain the reconstructed values ​​of three or more endpoints representing the geometric surface; for details, please refer to S803 to S804; on the other hand, S807 can be executed.

[0423] S803: Determine, according to the first identifier, whether the encoded bitstream includes the residual value of the second endpoint.

[0424] For example, the first identifier can be used to determine whether the encoder performs residual processing on the values ​​of the three endpoints used to represent the geometric surface. If the first identifier determines that the encoder performs residual processing on the values ​​of the three endpoints used to represent the geometric surface, then the encoded bitstream is determined to contain a residual between the value of the first endpoint and the value of the second endpoint (i.e., the residual value of the second endpoint), and S804 can be executed. If the first identifier determines that the encoder does not perform residual processing on the values ​​of the three endpoints used to represent the geometric surface, then the encoded bitstream is determined to not contain a residual between the value of the first endpoint and the value of the second endpoint, and S806 can be executed.

[0425] Among them, when the value of the first identifier is a preset identifier value (for example, the value of the first bit in the value of the first identifier is a preset identifier value such as "0"), the residual between the value of the first endpoint and the value of the second endpoint included in the encoded code stream can be determined; otherwise, the residual between the value of the first endpoint and the value of the second endpoint not included in the encoded code stream can be determined.

[0426] S804 , decoding the encoded code stream according to the decoding algorithm indicated by the first identifier to obtain a reconstructed value of the first endpoint, a reconstructed value of the residual value of the second endpoint, and a reconstructed value of the third endpoint.

[0427] Exemplarily, the encoding algorithm for encoding the value of the first endpoint, the value of the third endpoint, and the residual value of the second endpoint can be determined based on the first identifier; and the decoding algorithm corresponding to the encoding algorithm is used to decode the reconstructed value of the first endpoint, the reconstructed value of the third endpoint, and the reconstructed value of the residual value of the second endpoint from the encoded code stream.

[0428] S805 , adding the reconstructed value of the first endpoint and the reconstructed value of the residual value of the second endpoint to obtain the reconstructed value of the second endpoint.

[0429] S806 : Decode the reconstructed value of the first endpoint, the reconstructed value of the second endpoint, and the reconstructed value of the third endpoint from the encoded bitstream according to the decoding algorithm indicated by the first identifier.

[0430] S807 , decoding the encoded code stream according to the decoding algorithm indicated by the second identifier to obtain coordinate reconstruction values ​​of multiple projection points.

[0431] Exemplarily, the decoding algorithm indicated by the second identifier may be used to decode the coordinate reconstruction values ​​of multiple projection points from the encoded bitstream.

[0432] Exemplarily, when the decoding algorithm determined according to the second identifier is one, the decoding algorithm is adopted to decode the coordinate reconstruction values ​​of all dimensions of the projection point of each texture pixel point to the geometric surface from the encoded bitstream each time.

[0433] Exemplarily, when the encoding algorithm determined according to the second identifier is one, the decoding algorithm is adopted to decode the coordinate reconstruction value of one dimension of the projection point of each texture pixel point to the geometric surface from the encoded bitstream each time.

[0434] Exemplarily, when there are multiple encoding algorithms determined according to the second identifier, one decoding algorithm is adopted to decode the coordinate reconstruction value of one dimension of the projection point of each texture pixel point to the geometric surface from the encoded bitstream each time.

[0435] S808: Determine the type of the geometric surface according to the third identifier.

[0436] S809 , using an interpolation algorithm corresponding to the type of the geometric surface, interpolate the reconstructed values ​​of the three endpoints and the coordinate reconstructed values ​​of the multiple projection points to obtain reconstructed values ​​of multiple texture pixels.

[0437] Exemplarily, when the type of the geometric surface is a plane, a first interpolation algorithm may be used to interpolate the coordinate reconstruction values ​​of multiple projection points and the reconstruction values ​​of three endpoints representing the plane to obtain reconstruction values ​​of multiple texture pixels.

[0438] For example, the first interpolation algorithm can be expressed as the following formula (1): C = ept0 + (ept1 - ept0)u2 + (ept2 - ept1)v2 (1)

[0439] In formula (1), C is the reconstructed value of a texture pixel (such as RGB), u2 and v2 are the coordinate reconstructed values ​​of the projection point, and ept0, ept1, and ept2 are the reconstructed values ​​of the three endpoints.

[0440] It should be understood that the first interpolation algorithm may also be other algorithms, and this application does not impose any limitation on this.

[0441] Exemplarily, when the type of the geometric surface is an elliptical paraboloid, a second interpolation algorithm may be used to interpolate the coordinate reconstruction values ​​of multiple projection points and the reconstruction values ​​of the three endpoints representing the elliptical paraboloid to obtain reconstruction values ​​of multiple texture pixels.

[0442] For example, the second interpolation algorithm can be expressed as follows:

[0443] In formula (2) is the reconstruction value of a texture pixel, u2 and v2 are the coordinate reconstruction values ​​of the projection point, and a and b are the coefficients of the ellipse equation.

[0444] It should be understood that the second interpolation algorithm may also be other algorithms, and this application does not limit this.

[0445] Exemplarily, when the type of the geometric surface is an ellipsoid, a third interpolation algorithm may be used to interpolate the coordinate reconstruction values ​​of multiple projection points and the reconstruction values ​​for identifying three or more endpoints of the ellipsoid to obtain reconstruction values ​​of multiple texture pixels.

[0446] For example, the third interpolation algorithm can be expressed as follows:

[0447] In formula (3), x, y and z are the coordinate reconstruction values ​​of the projection point, a, b and c are the coefficients of the ellipse equation, and the reconstruction value of the texture pixel is (R, γ, λ).

[0448] It should be understood that the third interpolation algorithm may also be other algorithms, and this application does not impose any limitation on this.

[0449] 9 is a schematic diagram illustrating an exemplary texture image encoding process 900. The encoding process 900 illustrates another method of performing fitting processing on the pixel values ​​of a plurality of texture pixels in a texture image to obtain a geometric surface.

[0450] S901 , performing fitting processing on pixel values ​​of a plurality of texture pixels to obtain a plurality of candidate geometric surfaces, wherein at least two candidate geometric surfaces among the plurality of candidate geometric surfaces are of different types.

[0451] Exemplarily, all types of geometric surfaces can be traversed, and each time one type of geometric surface can be used to fit multiple texture pixels in the texture block to obtain multiple initial geometric surfaces; in this way, multiple types of initial geometric surfaces can be obtained, and each type of initial geometric surface can be multiple.

[0452] In one possible approach, the initial geometric surface may be used as a candidate geometric surface; thus, there are multiple candidate geometric surfaces of each type.

[0453] In one possible approach, the initial geometric surface with the smallest fitting error can be selected from multiple initial geometric surfaces of each type as a candidate geometric surface of that type. Thus, there is only one candidate geometric surface of each type. The process of determining the fitting error of the initial geometric surface can refer to the process of determining the fitting error of the candidate geometric surface described above and is not further described here.

[0454] S902: Determine the candidate geometric surface with the smallest corresponding coding error as the geometric surface.

[0455] For example, S902 may refer to the following steps S9021 to S90211:

[0456] S9021: Use the coding mode in the i-th coding mode combination to encode the coordinates of the projection points of multiple texture pixels to the j-th candidate geometric surface, where the initial value of i is 1 and the initial value of j is increased by 1.

[0457] Exemplarily, the number of candidate geometric faces is represented by H, and j is an integer between 1 and H.

[0458] S9022: Determine whether the residual between the value of the first endpoint and the value of the second endpoint is within a preset range.

[0459] S9023, encode the value of the first endpoint, the residual between the value of the first endpoint and the value of the second endpoint, and the value of the third endpoint.

[0460] S9024, encode the value of the first endpoint, the value of the second endpoint, and the value of the third endpoint respectively.

[0461] S9025: Determine the coding error corresponding to the i-th coding mode combination.

[0462] S9026, determine whether i is equal to M.

[0463] Exemplarily, when i is not equal to M, S9027 is executed; when i is equal to M, S9028 is executed.

[0464] S9027, add 1 to i.

[0465] For example, after executing S9027, you can return to execute S9021.

[0466] For example, S9021 to S9027 may refer to the description of S1 to S7 above, which will not be repeated here.

[0467] S9028: The minimum coding error is used as the coding error corresponding to the j-th candidate geometric surface.

[0468] For example, the coding errors corresponding to various coding modes can be calculated, and the minimum coding error can be used as the coding error corresponding to the j-th candidate geometric surface. The process of determining the coding errors corresponding to various coding modes can refer to the description of S8 above and will not be repeated here.

[0469] S9029, determine whether j is equal to H.

[0470] Exemplarily, when j is equal to H, S90211 is executed; when j is not equal to H, S90210 is executed.

[0471] S90210, add 1 to j and set i equal to 1.

[0472] For example, after executing S90210, you can return to execute S9021.

[0473] S90211: From the coding errors corresponding to the H candidate geometric surfaces, select the candidate geometric surface with the smallest coding error as the geometric surface.

[0474] Illustratively, after looping through S9021 to S90210, coding errors corresponding to H candidate geometric surfaces can be obtained; and from the coding errors corresponding to the H candidate geometric surfaces, the candidate geometric surface with the smallest coding error can be selected as the geometric surface.

[0475] S903, obtaining coordinates of a plurality of projection points in the geometric surface; wherein the plurality of projection points correspond one-to-one to pixel values ​​of a plurality of texture pixels, and the coordinates of the projection points include two or more dimensions.

[0476] S904: Obtain the values ​​of the three endpoints used to represent the geometric surface.

[0477] S905 , using a coding algorithm combination with the smallest coding error to encode the coordinates of multiple projection points.

[0478] It should be noted that the minimum coding error (i.e., the coding error corresponding to the candidate geometric surface with the minimum coding error) is determined in the above S90211. Therefore, the coding mode combination with the minimum coding error can be obtained; then, the coding mode combination with the minimum coding error is used to encode the coordinates of multiple projection points.

[0479] S906: Determine the residual between the value of the second endpoint and the value of the first endpoint as the residual value of the second endpoint; wherein the value of the first endpoint is used as the predicted value of the second endpoint.

[0480] S907: Determine whether the residual value of the second endpoint is within a preset range.

[0481] S908 , encoding the residual value of the second endpoint, the value of the first endpoint, and the value of the third endpoint.

[0482] S909 , respectively encode the value of the first endpoint, the value of the second endpoint, and the value of the third endpoint.

[0483] For example, S904 to S909 may refer to the description of S604 to S609 above, which will not be repeated here.

[0484] Among them, the decoding process corresponding to the encoding process 900 can be the above-mentioned decoding process 800, which will not be repeated here.

[0485] For example, multiple texture images were used for testing to compare the fitting error of the prior art when mapping the pixel values ​​of the texture pixels of the texture image to a straight line, and the fitting error of the present application when mapping the pixel values ​​of the texture pixels of the texture image to a plane (that is, fitting the pixel values ​​of the texture pixels to obtain a plane); as shown in Table 1 below:

[0486] Table 1

[0487] Table 1 shows the fitting errors of a texture block of size 8*8 when fitted into a plane using the method of the present invention, the fitting errors when fitted into a single line using the method of prior art 1, and the fitting errors when fitted into multiple lines using the method of prior art 2.

[0488] By comparison, it can be seen that the relative reduction ratio of the fitting error of the plane fitting method using the present invention compared to the fitting error of the single line fitting method using the prior art 1 (average value is 83.15%), and the relative reduction ratio of the fitting error of the plane fitting method using the present invention compared to the fitting error of the single line fitting method using the prior art 2 (average value is 62.43%). It can be seen that the fitting error of the present application is lower than the fitting error of the prior art. Therefore, the reconstructed values ​​of the texture pixels obtained by the present application are more accurate, and thus, at the same bit rate, the pixel values ​​of the reconstructed texture pixels are more accurate.

[0489] FIG10 is a schematic diagram showing exemplary pixel values ​​corresponding to projection points of fitting points onto straight lines and geometric surfaces.

[0490] In Figure 10, the black dots are fitting points, the plane formed by the three endpoints ept0, ept1, and ept2 is the best-fit plane for the pixel values ​​of the texture pixels, and the straight line is the best-fit straight line for the pixel values ​​of the texture pixels. According to mathematical theory, the best-fit straight line for a set of discrete points in space lies on the best-fit plane. This intuitively shows that the sum of the distances from the pixel values ​​of multiple texture pixels to the best-fit plane is less than the sum of the distances to the best-fit straight line. In other words, the fitting error of the pixel values ​​of the texture pixels in this application, which is based on geometric surface fitting, is less than the fitting error of the pixel values ​​of the texture pixels in prior art 1, which is based on single-line fitting.

[0491] 11 is a schematic diagram of an exemplary texture image encoding device 1100. The encoding device 1100 can be used to execute the method of the aforementioned embodiment. Therefore, the beneficial effects achieved by the encoding device 1100 can refer to the beneficial effects of the corresponding method provided above, and will not be repeated here.

[0492] 11 , the encoding apparatus 1100 may include:

[0493] The fitting processing module 1101 is used to perform fitting processing on the pixel values ​​of multiple texture pixels in the texture image to obtain a geometric surface;

[0494] A projection point acquisition module 1102 is configured to acquire coordinates of a plurality of projection points in a geometric surface; wherein the plurality of projection points correspond one-to-one to pixel values ​​of a plurality of texture pixels, and the coordinates of the projection points include two or more dimensions;

[0495] An endpoint acquisition module 1103 is used to obtain values ​​representing three or more endpoints of a geometric surface;

[0496] The encoding module 1104 is configured to encode the values ​​of three or more endpoints and the coordinates of multiple projection points to obtain an encoded bitstream.

[0497] Exemplarily, the three or more endpoints include: a first endpoint and a second endpoint,

[0498] The encoding module 1104 is specifically used to determine the residual between the value of the second endpoint and the value of the first endpoint as the residual value of the second endpoint; wherein the value of the first endpoint is used as the predicted value of the second endpoint; and encode the residual value of the second endpoint and the value of the first endpoint.

[0499] Exemplarily, the three or more endpoints also include: other endpoints; an encoding module 1104, specifically used to determine the residual between the value of the other endpoints and the value of the first endpoint as the residual value of the other endpoints; wherein the value of the first endpoint is used as the predicted value of the other endpoints; and encode the residual value of the other endpoints.

[0500] Exemplarily, the encoding module 1104 is specifically configured to encode the residual value of the second endpoint and the value of the first endpoint when the residual value of the second endpoint is within a preset range.

[0501] Exemplarily, the fitting processing module 1101 is specifically used to perform fitting processing on the pixel values ​​of multiple texture pixels in the texture image to obtain a geometric surface, including: performing fitting processing on the pixel values ​​of multiple texture pixels to obtain multiple candidate geometric surfaces; selecting a geometric surface from multiple candidate geometric surfaces according to the types of the multiple candidate geometric surfaces; wherein the type includes any one of the following: a plane, a curved surface or a geometric body.

[0502] Exemplarily, the fitting processing module 1101 is specifically used to determine, when multiple candidate geometric surfaces are of the same type, one of the N candidate geometric surfaces with the smallest corresponding fitting error as the geometric surface, where N is a positive integer; wherein the fitting error corresponding to the current candidate geometric surface among the multiple candidate geometric surfaces is determined based on the sum of the distances between the pixel values ​​of multiple texture pixels and the current candidate geometric surface.

[0503] Exemplarily, the fitting processing module 1101 is specifically used to determine that the candidate geometric surface with the smallest corresponding coding error is the geometric surface when the types of at least two candidate geometric surfaces among multiple candidate geometric surfaces are different; wherein the coding error corresponding to the current candidate geometric surface among the multiple candidate geometric surfaces is the error of the encoding used to represent the values ​​of three or more candidate endpoints of the current candidate geometric surface and the coordinates of multiple candidate projection points in the current candidate geometric surface, and the multiple candidate projection points correspond one-to-one to the pixel values ​​of multiple texture pixels.

[0504] Exemplarily, the encoding module 1104 is specifically configured to encode the coordinates of the plurality of projection points by adopting a combination of encoding algorithms with the smallest encoding error.

[0505] Exemplarily, the encoding module 1104 is specifically used to encode a dimension of the coordinates of a first projection point for a first texture pixel among multiple texture pixels according to a coding algorithm in the encoding algorithm combination with the minimum coding error when the encoding algorithm combination with the minimum coding error includes two or more encoding algorithms; wherein the first projection point corresponds to the pixel value of the first texture pixel.

[0506] Exemplarily, the encoding module 1104 is specifically used to encode all dimensions of the coordinates of the second projection point for a second texture pixel among multiple texture pixels according to a coding algorithm in the encoding algorithm combination with the minimum coding error when the encoding algorithm combination with the minimum coding error includes a coding algorithm; wherein the second projection point corresponds to the pixel value of the second texture pixel.

[0507] Exemplarily, the texture image includes a plurality of texture blocks, and the plurality of texture pixels belong to any one of the plurality of texture blocks.

[0508] Exemplarily, the texture image includes a plurality of texture blocks, each texture block includes a plurality of texture regions, and a plurality of texture pixels belong to any one of the texture regions.

[0509] 12 is a schematic diagram of an exemplary texture image decoding device 1200. The decoding device 1200 can be used to execute the method of the aforementioned embodiment. Therefore, the beneficial effects achieved by the decoding device 1200 can refer to the beneficial effects of the corresponding method provided above, which will not be repeated here.

[0510] 12 , the decoding apparatus 1200 may include:

[0511] The code stream acquisition module 1201 is used to obtain the encoded code stream;

[0512] A reconstruction information acquisition module 1202 is configured to decode the encoded bitstream to obtain reconstruction values ​​of three or more endpoints and coordinate reconstruction values ​​of multiple projection points; wherein the three or more endpoints are used to represent a geometric surface, the geometric surface includes multiple projection points, the multiple projection points correspond one-to-one to pixel values ​​of multiple texture pixels in the texture image, and the coordinate reconstruction values ​​include two or more dimensions;

[0513] The reconstruction value generating module 1203 is configured to generate reconstruction values ​​of a plurality of texture pixels in the texture image according to the reconstruction values ​​of three or more endpoints and the coordinate reconstruction values ​​of a plurality of projection points.

[0514] Exemplarily, the three or more endpoints include: a first endpoint and a second endpoint; a reconstruction information acquisition module 1202, specifically used to parse a first identifier from the encoded code stream; wherein the first identifier indicates whether to encode the residual value of the second endpoint, and the residual value of the second endpoint is the residual between the value of the second endpoint and the value of the first endpoint; when the value of the first identifier is a preset identifier value, decoding the encoded code stream to obtain a reconstructed value of the first endpoint and a reconstructed value of the residual value of the second endpoint; adding the reconstructed value of the first endpoint to the reconstructed value of the residual value of the second endpoint to obtain a reconstructed value of the second endpoint; wherein the reconstructed value of the first endpoint is used as a predicted value of the second endpoint.

[0515] Exemplarily, the three or more endpoints also include: other endpoints; a reconstruction information acquisition module 1202, which is specifically used to decode the encoded code stream when the value of the first identifier is a preset identifier value to obtain the reconstructed value of the residual value of the other endpoints; wherein the residual value of the other endpoints is the residual between the value of the other endpoints and the value of the first endpoint; the reconstructed value of the first endpoint is added to the reconstructed value of the residual value of the other endpoints to obtain the reconstructed value of the other endpoints.

[0516] Exemplarily, the reconstruction information acquisition module 1202 is specifically used to parse a second identifier from the encoded code stream; wherein the second identifier indicates a decoding algorithm corresponding to the coordinates of each projection point in the multiple projection points; and decode the encoded code stream according to the decoding algorithm indicated by the second identifier to obtain coordinate reconstruction values ​​of multiple projection points.

[0517] Exemplarily, the second identifier indicates multiple decoding algorithms, and the reconstruction information acquisition module 1202 is specifically configured to decode the decoded code stream using a decoding algorithm indicated by the second identifier to obtain reconstruction values ​​of one or more dimensions of the coordinates of each of the multiple projection points.

[0518] Exemplarily, the reconstruction value generation module 1203 is specifically configured to perform interpolation processing on the reconstruction values ​​of three or more endpoints and the coordinate reconstruction values ​​of multiple projection points to obtain reconstruction values ​​of multiple texture pixels.

[0519] Exemplarily, the reconstruction information acquisition module 1202 is further used to parse a third identifier from the code stream; wherein the third identifier indicates the type of the geometric surface; the reconstruction value generation module 1203 is specifically used to adopt the interpolation algorithm corresponding to the type of the geometric surface indicated by the third identifier to interpolate the reconstruction values ​​of three or more endpoints and the coordinate reconstruction values ​​of multiple projection points to obtain the reconstruction values ​​of multiple texture pixels.

[0520] In an example, FIG13 shows a schematic block diagram of a device 1300 according to an embodiment of the present application. The device 1300 may include: a processor 1301 and a transceiver / transceiver pin 1302 , and optionally, a memory 1303 .

[0521] The various components of the device 1300 are coupled together via a bus 1304, wherein the bus 1304 includes, in addition to a data bus, a power bus, a control bus, and a status signal bus. However, for the sake of clarity, all buses are referred to as bus 1304 in the figure.

[0522] Optionally, the memory 1303 may be used to store instructions in the aforementioned method embodiment. The processor 1301 may be used to execute the instructions in the memory 1303 and control the receiving pin to receive a signal and control the transmitting pin to send a signal.

[0523] The apparatus 1300 may be the electronic device or a chip of the electronic device in the above method embodiment.

[0524] Exemplarily, the electronic device may be a server or a terminal device.

[0525] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0526] The present application also provides a chip including one or more interface circuits and one or more processors. The one or more processors receive or send data via the one or more interface circuits. When the one or more processors execute computer instructions, the steps of the above-mentioned related methods are implemented. The interface circuit is a transceiver / transceiver pin 1302.

[0527] This embodiment further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the method in the above-mentioned embodiment.

[0528] This embodiment further provides a computer program product, which includes computer instructions. When the computer instructions are executed by a computer or a processor, the computer executes the above-mentioned related steps to implement the method in the above-mentioned embodiment.

[0529] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the chip to execute the methods in the above-mentioned method embodiments.

[0530] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0531] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0532] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0533] Units described as separate components may or may not be physically separate, and components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0534] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0535] Any content of each embodiment of this application, as well as any content of the same embodiment, can be freely combined. Any combination of the above content is within the scope of this application.

[0536] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0537] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner, or can be implemented by a processor executing a software instruction. The software instruction can be composed of corresponding software modules, and the software module can be stored in a random access memory (Random Access Memory, RAM), a flash memory, a read-only memory (Read Only Memory, ROM), an erasable programmable read-only memory (Erasable Programmable ROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), a register, a hard disk, a mobile hard disk, a read-only compact disc (CD-ROM) or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0538] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer-readable storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0539] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A texture image encoding method, characterized in that: The method comprises: Performing fitting processing on pixel values of a plurality of texture pixels in the texture image to obtain a geometric surface; Obtaining coordinates of a plurality of projection points in the geometric surface; wherein the plurality of projection points correspond one-to-one to pixel values of the plurality of texture pixels, and the coordinates of the projection points include two or more dimensions; Obtaining values of three or more endpoints representing the geometric surface; The values of the three or more endpoints are encoded, and the coordinates of the multiple projection points are encoded to obtain an encoded code stream.

2. The method according to claim 1, characterized in that The three or more endpoints include: a first endpoint and a second endpoint; encoding the value of the first endpoint and the value of the second endpoint includes: determining a residual between the value of the second endpoint and the value of the first endpoint as a residual value of the second endpoint; wherein the value of the first endpoint is used as a predicted value of the second endpoint; The residual value of the second endpoint and the value of the first endpoint are encoded.

3. The method according to claim 2, characterized in that The three or more endpoints further include: other endpoints; encoding the values of the other endpoints, including: Determining a residual between the value of the other endpoint and the value of the first endpoint as a residual value of the other endpoint; wherein the value of the first endpoint is used as a predicted value of the other endpoint; Encode the residual values of the other endpoints.

4. The method according to claim 2 or 3, characterized in that The encoding of the residual value of the second endpoint and the value of the first endpoint includes: When the residual value of the second endpoint is within a preset range, the residual value of the second endpoint and the value of the first endpoint are encoded.

5. The method according to any one of claims 1 to 4, characterized in that The fitting process is performed on the pixel values of a plurality of texture pixels in the texture image to obtain a geometric surface, comprising: Performing fitting processing on the pixel values of the plurality of texture pixels to obtain a plurality of candidate geometric surfaces; The geometric surface is selected from the multiple candidate geometric surfaces according to their types; wherein the types include any one of the following: a plane, a curved surface, or a geometric body.

6. The method according to claim 5, characterized in that The selecting the geometric surface from the multiple candidate geometric surfaces according to the types of the multiple candidate geometric surfaces includes: When the types of the multiple candidate geometric surfaces are the same, determining one of N candidate geometric surfaces with the smallest corresponding fitting error as the geometric surface, where N is a positive integer; The fitting error corresponding to the current candidate geometric surface among the multiple candidate geometric surfaces is determined according to the sum of distances between pixel values of multiple texture pixels and the current candidate geometric surface.

7. The method according to claim 5, characterized in that The selecting the geometric surface from the multiple candidate geometric surfaces according to the types of the multiple candidate geometric surfaces includes: When at least two of the multiple candidate geometric surfaces are of different types, determining the candidate geometric surface corresponding to the minimum coding error as the geometric surface; Among them, the encoding error corresponding to the current candidate geometric surface among the multiple candidate geometric surfaces is the error encoded to represent the values of three or more candidate endpoints of the current candidate geometric surface and the coordinates of multiple candidate projection points in the current candidate geometric surface, and the multiple candidate projection points correspond one-to-one to the pixel values of the multiple texture pixels.

8. The method according to any one of claims 1 to 7, characterized in that The texture image includes a plurality of texture blocks, and the plurality of texture pixels belong to any one of the plurality of texture blocks.

9. The method according to any one of claims 1 to 7, characterized in that The texture image includes a plurality of texture blocks, each texture block includes a plurality of texture regions, and the plurality of texture pixels belong to any one of the texture regions.

10. An encoded bitstream, characterized in that The encoded code stream is generated according to the texture image encoding method according to any one of claims 1 to 9.

11. An encoded bitstream, characterized in that The encoded code stream includes encoded values of three or more endpoints and encoded coordinates of multiple projection points; Among them, the three or more endpoints are used to represent a geometric surface, and the geometric surface is obtained by fitting the pixel values of multiple texture pixels in the texture image. The geometric surface includes the multiple projection points, and the multiple projection points correspond one-to-one to the pixel values of the multiple texture pixels. The coordinates of the projection points include two or more dimensions.

12. The encoded bitstream according to claim 11, wherein: The three or more endpoints include: a first endpoint and a second endpoint; the encoded code stream also includes a first identifier, the first identifier indicating whether to encode the residual value of the second endpoint, and the residual value of the second endpoint is the residual between the value of the first endpoint and the value of the second endpoint.

13. The encoded bitstream according to claim 11 or 12, characterized in that The encoded code stream further includes a second identifier, which indicates an encoding algorithm corresponding to the coordinates of each projection point in the multiple projection points.

14. The encoded bitstream according to claim 13, wherein: The second identifier indicates an encoding algorithm corresponding to each dimension of the coordinates of each projection point in the multiple projection points.

15. The encoded bitstream according to any one of claims 11 to 14, characterized in that The encoded code stream further includes a third identifier, which indicates the type of the geometric surface.

16. A texture image decoding method, characterized in that: The method comprises: Obtain the encoded code stream; Decoding the encoded bitstream to obtain reconstruction values of three or more endpoints and coordinate reconstruction values of a plurality of projection points, wherein the three or more endpoints are used to represent a geometric surface, the geometric surface includes the plurality of projection points, the plurality of projection points correspond one-to-one to pixel values of a plurality of texture pixels in the texture image, and the coordinate reconstruction values include two or more dimensions; Reconstructed values of a plurality of texture pixels in the texture image are generated according to the reconstructed values of the three or more endpoints and the coordinate reconstructed values of the plurality of projection points.

17. The method according to claim 16, characterized in that The three or more endpoints include: a first endpoint and a second endpoint; decoding the encoded bitstream to obtain a reconstructed value of the first endpoint and a reconstructed value of the second endpoint, comprising: Parsing a first identifier from the encoded bitstream; wherein the first identifier indicates whether to encode a residual value of the second endpoint, the residual value of the second endpoint being a residual between a value of the second endpoint and a value of the first endpoint; When the value of the first identifier is a preset identifier value, decoding the encoded code stream to obtain a reconstructed value of the first endpoint and a reconstructed value of the residual value of the second endpoint; The reconstructed value of the first endpoint is added to the reconstructed value of the residual value of the second endpoint to obtain the reconstructed value of the second endpoint; wherein the reconstructed value of the first endpoint is used as the predicted value of the second endpoint.

18. The method according to claim 17, characterized in that The three or more endpoints further include: other endpoints; decoding the encoded code stream to obtain a reconstructed value of the other endpoint, including: When the value of the first identifier is a preset identifier value, decoding the encoded bitstream to obtain a reconstructed value of the residual value of the other endpoint; wherein the residual value of the other endpoint is a residual between the value of the other endpoint and the value of the first endpoint; The reconstructed value of the first endpoint is added to the reconstructed value of the residual value of the other endpoint to obtain the reconstructed value of the other endpoint.

19. The method according to any one of claims 16 to 18, characterized in that The decoding of the encoded code stream to obtain coordinate reconstruction values of a plurality of projection points includes: Parsing a second identifier from the encoded bitstream; wherein the second identifier indicates a decoding algorithm corresponding to the coordinates of each projection point in the multiple projection points; The encoded code stream is decoded according to the decoding algorithm indicated by the second identifier to obtain coordinate reconstruction values of the multiple projection points.

20. The method according to claim 19, wherein The second identifier indicates a plurality of decoding algorithms, and decoding the encoded code stream according to the decoding algorithm indicated by the second identifier to obtain coordinate reconstruction values of the plurality of projection points includes: The encoded code stream is decoded using a decoding algorithm indicated by the second identifier to obtain reconstructed values of one or more dimensions of the coordinates of each of the multiple projection points.

21. The method according to any one of claims 16 to 20, characterized in that The determining, based on the reconstruction values of the three or more endpoints and the coordinate reconstruction values of the multiple projection points, the reconstruction values of the multiple texture pixels in the texture image includes: Interpolation processing is performed on the reconstructed values of the three or more endpoints and the coordinate reconstructed values of the multiple projection points to obtain reconstructed values of the multiple texture pixels.

22. The method according to claim 21, characterized in that The method further comprises: Parsing a third identifier from the encoded code stream; wherein the third identifier indicates the type of the geometric surface; The interpolation processing is performed on the reconstruction values of the three or more endpoints and the coordinate reconstruction values of the multiple projection points to obtain the reconstruction values of the multiple texture pixels, including: The reconstructed values of the three or more endpoints and the coordinate reconstructed values of the multiple projection points are interpolated using an interpolation algorithm corresponding to the type of the geometric surface indicated by the third identifier to obtain reconstructed values of the multiple texture pixels.

23. An encoder, characterized in that The method comprises a processing circuit for executing the method according to any one of claims 1 to 9.

24. A decoder, characterized in that comprising processing circuitry for performing a method according to any one of claims 16 to 22.

25. A computer-readable storage medium, characterized in that The computer-readable storage medium stores the encoded code stream according to any one of claims 10 to 15.

26. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when run on a computer or a processor, enables the computer or the processor to execute the method according to any one of claims 1 to 9, or the method according to any one of claims 16 to 22.

27. A computer program product, characterized in that The computer program product comprises computer instructions, and when the computer instructions are executed by a computer or a processor, the steps of the method according to any one of claims 1 to 9 are performed, and the steps of the method according to any one of claims 16 to 22 are performed.

28. A device for storing a code stream, characterized in that: The device comprises: a receiver and at least one storage medium, The receiver is configured to receive the encoded code stream according to any one of claims 10 to 15; The at least one storage medium is used to store the encoded code stream.

29. A device for transmitting a code stream, characterized in that: The device comprises: a transmitter and a receiver, The receiver is configured to receive the encoded code stream according to any one of claims 10 to 15; The transmitter is configured to transmit the encoded code stream to an end-side device via a transmission medium.

30. A device for transmitting a code stream, characterized in that: The device includes: a transmitter and at least one storage medium, The at least one storage medium is used to store the encoded code stream according to any one of claims 10 to 15; The transmitter is configured to obtain the encoded code stream from the storage medium and send the code stream to the terminal device via a transmission medium.

31. A system for distributing code streams, characterized in that: The system comprises: At least one storage medium for storing at least one encoded bitstream according to any one of claims 10 to 15; The streaming media device is configured to obtain a target code stream from the at least one storage medium and send the target code stream to the terminal side device, wherein the streaming media device includes a content server or a content distribution server.

Images