Compression and decompression of 3D scene representations

Abstract

A computer-implemented method and apparatus for compressing a representation of a three-dimensional scene and decompressing the compressed representation of the scene. Compressing the representation of the scene can include detecting a subset of position parameters included in the scene, indicative of locations of discontinuities in a measure of a characteristic of the scene, and storing the subset of the position parameters and the measure of the characteristic of the scene corresponding to the subset of the position parameters as the compressed representation of the scene. The compressed representation of the three-dimensional scene can include a first measure of the characteristic of the scene corresponding to a first plurality of position parameters indicative of a first location within the scene. Decompressing the compressed representation of the scene can include determining a second measure of the characteristic of the scene corresponding to a second plurality of position parameters indicative of a second location within the scene.

Description

[Technical field]

[0001] This disclosure relates to computer apparatus and methods for compressing and decompressing representations of three-dimensional scenes, which may find particular application in the rendering of three-dimensional scenes to generate images for display. [Background technology]

[0002] Rendering a scene to generate an image for display finds a variety of applications in areas such as generating visual effects for display at a live audience event or for inclusion in recorded media such as television programs and movies. Additionally or alternatively, rendering of a three-dimensional scene finds application in generating graphics for video games and simulators. The three-dimensional scene to be rendered may include, for example, a computer-generated virtual scene. Additionally or alternatively, the rendered scene may include at least some components of a photographed and / or measured real-world scene. In at least some applications, rendering facilitates the creation of extended reality (XR) experiences, including augmented reality (AR), mixed reality (MR), and virtual reality (VR).

[0003] Rendering a three-dimensional scene to generate an image for display typically requires a representation of one or more properties of the three-dimensional scene represented for a number of different locations within the scene. For example, a representation may be provided of properties related to the geometry of objects in the scene, such as the location, shape and / or orientation of surfaces in the scene. Using such representations, the appearance of objects in the scene when viewed from a particular viewing position can be determined, and an image of the scene as viewed from a particular viewing position can be rendered.

[0004] In at least some rendering applications, it may be desirable to compress a representation of a three-dimensional scene to produce a compressed representation that consumes less memory than a full uncompressed representation, and further, it may be desirable to decompress the compressed representation to reconstruct an approximation of the original representation of the scene.

[0005] It is in this context that the subject matter contained in this application was conceived. Summary of the Invention [Means for solving the problem]

[0006] It is understood that representations of three-dimensional scenes often contain relatively low frequency information, while high frequency features can be discarded without significant loss of information. For example, characteristics of three-dimensional scenes often undergo abrupt transitions at edges of various objects within the scene, and, for example, regions between such abrupt transitions may be relatively uniform or vary relatively uniformly as a function of position. It is further understood that representations of three-dimensional scenes can be efficiently compressed by identifying discontinuities in a measure of the scene characteristics and storing the locations of the discontinuities, along with specific measures at the locations of the discontinuities, in the compressed representation of the scene. Such compression can achieve relatively high compression ratios while retaining much of the important information in the scene representation. For example, when compressing a representation of a scene, by retaining low frequency information associated with discontinuities in the scene characteristics and discarding high frequency information, sufficient information can be retained so that the compressed representation can be decompressed and used to render an image of the scene without significant loss of quality.

[0007] It has also been found that the compression and decompression methods disclosed herein are particularly suitable for implementation in a graphics processing unit (GPU). In particular, the methods disclosed herein are particularly suitable for parallelization, allowing multiple processing cores and / or multiple processing units to perform aspects of the method steps disclosed herein in parallel with each other. This can significantly improve the speed at which compression and decompression can be performed in accordance with the methods disclosed herein. This can be particularly advantageous for real-time rendering applications.

[0008] According to a first aspect of the present disclosure, there is provided a computer-implemented method for compressing a representation of a three-dimensional scene, the representation including a measure of a property of the scene corresponding to a plurality of different location parameters indicative of a position within the scene, the method comprising: detecting a first subset of the plurality of location parameters indicative of a location of a discontinuity in the measure of the property of the scene, and storing the first subset of the plurality of location parameters and the measure of the property of the scene corresponding to the first subset of the plurality of location parameters as a compressed representation of the scene.

[0009] In some examples, each of the multiple different position parameters can completely define a three dimensional position within the scene. In at least some examples, each of the multiple different position parameters can define a one or two dimensional position within the scene.

[0010] The position parameters may indicate two-dimensional positions. For example, a scale of a characteristic of a scene may correspond to a number of different two-dimensional positions in the scene. Each position parameter may include two sets of coordinates. The two coordinates in each set of two coordinates may relate to two different dimensions. That is, each position parameter may be expressed as a set of two-dimensional coordinates. For example, each scale of a characteristic of a scene may be associated with a different set of two-dimensional coordinates. The set of two-dimensional coordinates may include, for example, a position on an x-axis and a position on a y-axis (or, equivalently, a position on a u-axis and a position on a v-axis). The positions on the x-axis and the positions on the y-axis may correspond to positions of pixels in an image rendered based on the representation of the three-dimensional scene. Similarly, the positions on the x-axis and the positions on the y-axis may correspond to two-dimensional viewing angles or directions from a viewing position, respectively. The viewing position represents a relative viewing position to which an image is rendered based on the representation of the three-dimensional scene.

[0011] In at least some examples, the characteristics of the scene can include a three-dimensional position, and the measure of the characteristics of the scene can include a position of a feature (e.g., a surface, etc.) within the three-dimensional scene. In at least some examples, the three-dimensional position can be expressed as a distance from a viewing position.

[0012] The scene characteristics may be characteristics of surfaces in the scene. For example, the scene characteristics may relate to the geometry, material properties, and / or optical properties of surfaces in the scene. Additionally or alternatively, the scene characteristics may relate to illumination, lighting, and / or shadow properties of the scene expressed as a function of the position parameters.

[0013] In at least some examples, measures of a property of a scene that correspond to different location parameters indicative of a location within the scene can be represented as an image, for example, each measure of the property of the scene can be represented as a pixel intensity, and the location parameter to which the measure corresponds can determine the location within the image at which the measure is represented as a pixel intensity.

[0014] Detecting a first subset of the plurality of location parameters indicative of locations of discontinuities in the scale of the characteristic of the scene may include performing one or more edge detection processes to detect the discontinuities For example, if the scale is represented as an image, any suitable edge detection process may be performed on the image to identify locations of discontinuities in the scale.

[0015] The compressed representation of the scene may be fully represented as a subset of the plurality of distinct position parameters included in the uncompressed representation and measures of characteristics of the scene corresponding to the subset of the plurality of distinct position parameters. The subset of the plurality of distinct position parameters includes at least the detected first subset. In at least some examples, the subset of the plurality of distinct position parameters representing the compressed representation may include one or more additional subsets of position parameters (in addition to the detected first subset) and measures of characteristics of the scene corresponding to the one or more additional subsets.

[0016] The compressed representation of a scene may not include additional information that defines how a measure of a characteristic of the scene varies among the positions indicated by a subset of position parameters included in the compressed representation. For example, the method of compressing the representation may not include fitting a function to the measure of the characteristic of the scene. The compressed representation may not include function parameters that define a function that represents the variation of the measure of the characteristic of the scene.

[0017] By fully representing the compressed representation of a scene as a subset of the different position parameters contained in the uncompressed representation and measures of scene properties corresponding to the subset of the different position parameters, a high level of compression can be achieved, and such a compression method further allows the compressed representation to be decompressed to recreate the original representation with a high degree of accuracy.

[0018] The representation of the three-dimensional scene is suitable for use in rendering an image of the scene. That is, the characteristics of the scene may be characteristics used to render an image of the three-dimensional scene. Different position parameters in the scene may correspond to different pixels in an image rendered based on the representation of the scene. For example, there may be a one-to-one mapping between the position parameters and pixels in an image rendered using the representation of the scene.

[0019] The measures of the properties of the scene may include measures of properties of surfaces within the scene corresponding to a number of different position parameters.

[0020] The property may relate, for example, to material and / or optical properties of surfaces in the scene expressed in terms of a number of different positional parameters. For example, the measure of the property of the scene may be indicative of the reflectance of the surfaces in the scene, expressed, for example, as an albedo. The reflectance and / or albedo measures may be expressed, for example, for a number of different colors (e.g., red, green, blue). In at least some examples, the measure of the property of the scene may include at least one of a red albedo, a blue albedo, and a green albedo.

[0021] In at least some examples, the measure of the scene characteristic may be indicative of optical distortion caused by a surface in the scene. In at least some examples, the measure of the scene characteristic may be indicative of a color of a surface in the scene.

[0022] The measures of properties of surfaces in the scene may include measures indicative of the geometry of surfaces in the scene corresponding to a number of different position parameters.

[0023] For example, the properties may relate to the position, shape, and / or orientation of the surface within the scene. In at least some examples, the properties may relate to the distance of the surface from a reference position or plane (e.g., a viewing position). In at least some examples, the properties may relate to the orientation of the surface within the scene. The surface orientation may be expressed, for example, as a direction of a surface normal. A measure of the property of the scene may include, for example, a measure of the surface normal direction corresponding to a number of different position parameters (which may relate, for example, to different pixels in a rendered image, or equivalently, to different viewing directions from a viewing position).

[0024] The different position parameters may correspond to different viewing directions originating from the viewing position. Each of the measures of the scene properties may correspond to a viewing direction of the plurality of viewing directions.

[0025] The different viewing directions may be equated to different light rays each passing through the viewing position, which may correspond to light rays used in rendering techniques such as ray casting, for example.

[0026] The different viewing directions may be characterized in two dimensions. For example, the different viewing directions may relate to different positions on the x-axis and y-axis (or equivalently the u-axis and v-axis). The x-axis and y-axis may relate directly to the position of a pixel in an image that is rendered based on the representation of the scene. Additionally or alternatively, the different viewing directions may be represented as different angles of the viewing direction relative to a viewing position. Each viewing direction may be represented as a two-dimensional angle relative to some reference direction. For example, each viewing direction may be represented as a polar angle and an azimuth angle.

[0027] The representation of the three-dimensional scene can include a depth map, and the measures of the scene properties can include distances from a viewing position to surfaces in the three-dimensional scene along a number of different viewing position directions.

[0028] The distance may include the distance from the viewing position along each viewing direction to the nearest surface in the scene. For example, the distance can be thought of as the distance along a ray that originates at the viewing position and extends along each viewing direction, the distance being the distance along the ray until it first intersects a surface in the scene.

[0029] In at least some examples, the distances may additionally or alternatively include at least some distances to occluded surfaces in the scene when viewed from the viewing position. For example, at least some distances may include distances from the viewing position to second, third, or subsequent surfaces intersected by rays extending in respective viewing directions.

[0030] Detecting a first subset of the plurality of location parameters indicative of a location of a discontinuity in a measure of the characteristic of the scene may include determining a derivative of the measure of the characteristic of the scene with respect to changes in the location parameters, and determining the first subset of the plurality of location parameters as those location parameters for which a magnitude of the determined derivative is greater than a derivative threshold.

[0031] The derivative of a scale of a characteristic of a scene may include a combination of a derivative of the scale determined with respect to a change in position in one dimension (e.g., the x-direction) and a derivative of the scale determined with respect to a change in position in two dimensions (e.g., the y-direction). The derivative may be expressed as an absolute derivative, or equivalently, as the magnitude of the determined derivative.

[0032] The derivatives may be determined by performing an operation on scales of the scene characteristics corresponding to the different location parameter characteristics represented as images, for example, determining the derivatives may include performing a convolution of the scales represented as images with a kernel.

[0033] The derivative of the measure of the characteristic of the scene with respect to the change in the position parameter may include a first derivative with respect to the change in the position parameter. Determining the derivative may include performing a convolution of the measure of the characteristic of the scene represented as an image with a kernel such as, for example, a Sobel operator, a Scharr operator, and / or a Prewitt operator.

[0034] The derivative threshold may be a predetermined threshold. Additionally or alternatively, the derivative threshold may be determined in response to one or more characteristics of the scene representation. For example, the derivative threshold may be determined for each scene representation and set to achieve a desired compression ratio.

[0035] Determining a derivative of the measure of the characteristic of the scene may include determining a second derivative of the measure of the characteristic of the scene with respect to a change in the position parameter.

[0036] Determining the derivative may include determining a first derivative of the measure of the characteristic of the scene with respect to the change in the position parameter, and further determining a first derivative of the determined first derivative of the measure of the characteristic of the scene to calculate a second derivative of the measure of the characteristic of the scene with respect to the change in the position parameter. For example, a kernel such as a Sobel operator, a Schall operator, and / or a Prewitt operator may be applied twice to calculate the second derivative.

[0037] Determining the derivative may include performing a convolution of a measure of the characteristic of the scene represented as an image with a kernel configured to calculate a second derivative. The kernel may include, for example, a Laplacian kernel.

[0038] The computer-implemented method may further include determining a derivative threshold as a function of the number of location parameters for which the magnitude of the determined derivative is greater than the derivative threshold.

[0039] The derivative threshold may be determined to achieve a desired compression ratio. For example, it may be desirable for the compressed representation of the scene to contain a target percentage or less of the data points in the original uncompressed representation of the scene. The derivative threshold may be determined such that the number of location parameters whose determined derivative magnitude is greater than the derivative threshold is a target percentage or less of the total number of distinct location parameters included in the uncompressed representation of the scene.

[0040] The computer-implemented method may further include sampling the different positional parameters to determine a second subset of the plurality of positional parameters, and storing the first subset of the plurality of positional parameters and the measure of the characteristic of the scene at the first subset of the plurality of positional parameters as well as the second subset of the plurality of positional parameters and the measure of the characteristic of the scene corresponding to the second subset of the plurality of positional parameters as the compressed representation of the scene.

[0041] Sampling the plurality of different location parameters may include performing a random sampling process. Sampling the plurality of different location parameters may include sampling the different location parameters at uniform intervals. Sampling the plurality of different location parameters may include sampling location parameters that are in intervals between a first subset of the detected plurality of location parameters.

[0042] Sampling a plurality of different location parameters and further including a second subset of the plurality of location parameters and a measure of the characteristic of the scene corresponding to the second subset of location parameters can provide additional information about the characteristic of the scene in the spatial domain between discontinuities in the characteristic measures. For example, the additional sampling can provide information about the shape and / or gradient of the variation of the characteristic measure in the spatial domain between discontinuities in the characteristic measures. Such additional information can improve the accuracy with which decompression of the compressed representation matches the original representation.

[0043] According to a second aspect of the present disclosure, there is provided a computer-implemented method for decompressing a compressed representation of a three-dimensional scene, the compressed representation including a first measure of a characteristic of the scene corresponding to a plurality of first location parameters indicative of a first location within the scene, the method including determining a second measure of the characteristic of the scene corresponding to a plurality of second location parameters indicative of a second location within the scene, the determining the second measure of the characteristic of the scene corresponding to the plurality of second location parameters including, for each second location parameter, determining a subset of the plurality of first location parameters indicative of a first location located in a vicinity of the second location indicated by the second location parameters, interpolating the first measure of the characteristic of the scene corresponding to the determined subset of the plurality of first parameters to determine the second measure of the characteristic of the scene corresponding to the second location parameters, and forming a decompressed representation of the three-dimensional scene, the decompressed representation including the first measure of the characteristic of the scene corresponding to the plurality of first location parameters and the determined second measure of the characteristic of the scene corresponding to the plurality of second location parameters.

[0044] In some examples, each of the plurality of first location parameters can completely define a three-dimensional position of a first location indicated by the first location parameters, and similarly, each of the plurality of second location parameters can completely define a three-dimensional position of a second location indicated by the second location parameters.

[0045] In at least some examples, each of the plurality of first location parameters can define a first location indicated by the first location parameter in one or two dimensions, and similarly, each of the plurality of second location parameters can define a second location indicated by the second location parameter in one or two dimensions.

[0046] Each of the position parameters may indicate a two-dimensional position. For example, a first scale and a second scale of the scene characteristic may correspond to a number of different two-dimensional positions in the scene. Each position parameter may include two sets of coordinates. The two coordinates in each set of two coordinates may relate to two different dimensions. That is, each position parameter may be expressed as a set of two-dimensional coordinates. For example, each scale of the scene characteristic may be associated with a different set of two-dimensional coordinates. The set of two-dimensional coordinates may include, for example, a position on an x-axis and a position on a y-axis (or equivalently, a position on a u-axis and a position on a v-axis). The positions on the x-axis and the positions on the y-axis may correspond to positions of pixels in an image rendered based on the representation of the three-dimensional scene. Similarly, the positions on the x-axis and the positions on the y-axis may correspond to two-dimensional viewing angles or viewing directions from a viewing position. The viewing position may represent a relative viewing position to which an image is rendered based on the representation of the three-dimensional scene.

[0047] In at least some examples, the characteristic of the scene may include a three-dimensional position, and the first and second measures of the characteristic of the scene may include a position of a feature (e.g., a surface, etc.) within the three-dimensional scene. In at least some examples, the three-dimensional position may be expressed as a distance from a viewing position.

[0048] The scene characteristics may be characteristics of surfaces in the scene. For example, the scene characteristics may relate to the geometry, material properties, and / or optical properties of surfaces in the scene. Additionally or alternatively, the scene characteristics may relate to illumination, lighting, and / or shadow properties of the scene expressed as a function of the position parameters.

[0049] In at least some examples, the first and second measures of the characteristic of the scene corresponding to the first and second plurality of location parameters, respectively, can be represented as an image. For example, each measure of the characteristic of the scene can be represented as a pixel intensity, and the location parameter to which the measure corresponds can determine a location in the image at which the measure is represented as a pixel intensity. The compressed representation of the scene may correspond to only a portion of the pixels of the image. A method of decompressing the compressed representation can reconstruct the complete image. That is, the second measures of the characteristic of the scene corresponding to the second plurality of location parameters can be thought of as pixels of the image that are missing in the compressed representation.

[0050] The decompressed representation of the three-dimensional scene is suitable for use in rendering an image of the scene. That is, the characteristics of the scene may be characteristics used in rendering the image of the three-dimensional scene. The first and second location parameters may correspond to different pixels in an image rendered based on the representation of the scene. The first location parameters included in the compressed representation may correspond to a first portion of the pixels of the rendered image. The second location parameters may correspond to a remaining portion of the pixels of the rendered image for a measure of the characteristics of the scene not included in the compressed representation. The method of decompression functions to determine a measure of the characteristics of the scene for the remaining pixels of the rendered image such that the measure of the characteristics is available for all pixels.

[0051] Each second position may represent a position in the parameter space for which a measure of a property of the scene is to be determined. For example, the parameter space may represent a range of discrete viewing directions from the viewing position for which a measure of a property of the scene is to be determined (e.g., to render an image of the scene). The range of discrete viewing directions may correspond to pixels of the image to be rendered. The first position and the first position parameters may include a portion of the range of discrete viewing directions. The second position and the second position parameters may include the remaining viewing directions included in the range of discrete viewing directions. That is, the second position and the second position parameters may include viewing directions included in the range of discrete viewing directions not corresponding to any of the first position parameters.

[0052] The subset of the plurality of first location parameters indicating the first locations located near the second location indicated by the second location parameters may include the first locations located near the second location in the parameter space defined by the location parameters. For example, as described above, each of the location parameters may define a location in one, two, or three dimensions. In an example where the location parameters define a location in two dimensions, determining the subset of the plurality of first location parameters may include determining a subset of the first locations located near the second location in the two dimensions defined by the location parameters. For example, if the location parameters relate to different viewing directions from a viewing position, the first locations can be determined where the second locations correspond to viewing directions near the corresponding viewing directions. Similarly, if the location parameters relate to the locations of pixels in a two-dimensional pixel array (which may be defined by x and y locations), the first locations can be determined where the second locations correspond to pixel locations near the corresponding pixel locations.

[0053] In an example in which the position parameters define a three-dimensional position, determining a subset of the plurality of first position parameters may include determining a subset of the first positions that are located near the second positions in the three-dimensional space.

[0054] Determining a subset of the plurality of first location parameters indicative of first locations located near the second location indicated by the second location parameters may include determining a subset of first locations that are closest to the second location (in the parameter space defined by the location parameters). For example, first locations that are less distant from the second location (in the parameter space defined by the location parameters) than the distances of other parameters of the plurality of first location parameters from the second location may be determined. In other words, determining a subset of the plurality of first location parameters may include determining first locations that are nearest neighbors of the second location (in the parameter space defined by the location parameters).

[0055] Interpolating the first measure of the characteristic of the scene corresponding to the subset of the plurality of first parameters to determine the second measure of the characteristic of the scene corresponding to the second location parameter may include determining the second measure of the characteristic of the scene depending on the subset of the plurality of first parameters, the second location parameter, and the first measure of the characteristic of the scene corresponding to the subset of the plurality of first parameters. For example, a function of a certain form (e.g., a linear function) may be fit to the subset of the plurality of first parameters and the first measure of the characteristic of the scene corresponding to the subset of the plurality of first parameters. The fitted function may be used to determine the second measure of the characteristic of the scene based on the second location parameter.

[0056] The first measure and the determined second measure of the property of the scene may comprise measures of a property of a surface within the scene corresponding to a plurality of first and second location parameters, respectively.

[0057] The property may relate to, for example, material and / or optical properties of surfaces in the scene expressed in terms of a number of different positional parameters. For example, the measure of the property of the scene may be indicative of the reflectance of the surfaces in the scene, expressed, for example, as an albedo. The reflectance and / or albedo measure may be expressed, for example, for a number of different colors (e.g., red, green, blue). In at least some examples, the measure of the property of the scene may include at least one of a red albedo, a blue albedo, and a green albedo.

[0058] In at least some examples, the measure of the scene characteristic may be indicative of an optical distortion of a surface in the scene. In at least some examples, the measure of the scene characteristic may be indicative of a color of a surface in the scene.

[0059] The measures of properties of surfaces in the scene may include measures that indicate the geometry of surfaces in the scene.

[0060] For example, the properties may relate to the position, shape, and / or orientation of the surface within the scene. In at least some examples, the properties may relate to the distance of the surface from a reference position or plane (e.g., a viewing position). In at least some examples, the properties may relate to the orientation of the surface within the scene. The surface orientation may be expressed, for example, as a direction of a surface normal. A measure of the property of the scene may include, for example, a measure of the surface normal direction corresponding to a number of different position parameters (which may relate, for example, to different pixels in a rendered image, or equivalently, to different viewing directions from a viewing position).

[0061] The first and second position parameters may correspond to a plurality of different viewing directions from the viewing position, and each of the first and second measures of the characteristic of the scene may correspond to a viewing direction of the plurality of viewing directions.

[0062] The different viewing directions can be considered as equivalent to different light rays each passing through the viewing position. The different light rays may correspond to light rays in a rendering technique such as ray casting.

[0063] The different viewing directions can be characterized in two dimensions. For example, the different viewing directions can relate to different positions on the x-axis and the y-axis. The x-axis and the y-axis can relate directly to the position of a pixel in an image that is rendered based on the representation of the scene. Additionally or alternatively, the different viewing directions can be represented as different angles of the viewing direction relative to the viewing position. Each viewing direction can be represented as a two-dimensional angle relative to some reference direction. For example, each viewing direction can be represented as a polar angle and an azimuth angle.

[0064] The compressed representation of the three-dimensional scene may include a portion of a depth map, where the first and second measures of the property of the scene include distances from a viewing position to surfaces in the three-dimensional scene along a number of different viewing directions.

[0065] The distance may include the distance from the viewing position to the nearest surface in the scene along each viewing direction. For example, the distance may be considered as the distance along a ray that originates at the viewing position and extends along each viewing direction. The distance is the distance along the ray until the ray first intersects a surface in the scene.

[0066] In at least some examples, the distances may additionally or alternatively include at least some distances to occluded surfaces in the scene when viewed from the viewing position, for example, at least some distances may include distances to second, third, or subsequent surfaces intersected by rays extending in respective viewing directions from the viewing position.

[0067] Interpolating the first measures corresponding to a subset of the plurality of first parameters to determine a second measure of the characteristic of the scene corresponding to the second position parameter may include linear interpolation.

[0068] Interpolating the first measures corresponding to a subset of the plurality of first parameters to determine a second measure of the characteristic of the scene corresponding to the second location parameter may include determining a weight associated with each of the subset of the plurality of first location parameters in response to a distance between a first location indicated by the first location parameter and a second location indicated by the second location parameter, and determining a weighted average of the first measures of the characteristic of the scene corresponding to the subset of the plurality of first parameters. The weighted average may be determined using the determined weight associated with each of the subset of the plurality of first location parameters.

[0069] The distance between a first location indicated by the first location parameter and a second location indicated by the second location parameter may include a distance in a parameter space defined by the location parameters. For example, if the location parameters define a two-dimensional location, the distance between the first location and the second location may include a distance in the two-dimensional space. If the location parameters define a three-dimensional location, the distance between the first location and the second location may include a distance in the three-dimensional space.

[0070] A first location having a smaller distance to the second location may be assigned a larger weight than a first location having a larger distance to the second location. For example, the determined weight may be inversely proportional to the distance between the first location and the second location. That is, a measure of the scene characteristic corresponding to a first location that is closer to the second location may have a greater influence on the determined second measure of the scene characteristic corresponding to the second location parameter.

[0071] Interpolating the first measures corresponding to a subset of the plurality of first parameters to determine a second measure of the characteristic of the scene corresponding to the second location parameter may include performing an inverse distance weighting.

[0072] Determining a subset of the plurality of first position parameters indicative of a first position located near the second position indicated by the second position parameter may include dividing a parameter space characterized by the first position parameters into a plurality of segments within which each of the first position parameters falls, identifying at least one adjacent segment within which the second position parameter falls and / or adjacent to the segment within which the second position parameter falls, and determining the subset of the plurality of first position parameters as at least some of the plurality of first position parameters located within the at least one adjacent segment.

[0073] As discussed above, the location parameters can define a location in one, two, or three dimensions. Thus, the parameter space characterized by the first location parameter can include a corresponding parameter space in one, two, or three dimensions. For example, if the location parameters define a location in two dimensions, the parameter space is a corresponding two-dimensional parameter space. If the location parameters define a location in three dimensions, the parameter space is a corresponding three-dimensional parameter space.

[0074] In some examples, the second location parameter may be located (or may fall within) a segment of the parameter space that includes one or more of the plurality of first location parameters. In such examples, the at least one proximate segment may be identified to include the same segment in which the second location parameter falls. In some examples, the at least one proximate segment may be identified to include at least one segment adjacent to the same segment in which the second location parameter falls.

[0075] Determining the subset of the plurality of first position parameters as at least a portion of the plurality of first position parameters located in the at least one proximal segment may include selecting all of the first position parameters located in the at least one proximal segment. Alternatively, determining the subset of the plurality of first position parameters as at least a portion of the plurality of first position parameters located in the at least one proximal segment may include selecting one or more first position parameters in each of the proximal segments (e.g., the first position parameters in each proximal segment represent positions closest to the second position).

[0076] Dividing the parameter space characterized by the first location parameters into a plurality of segments into which each of the first location parameters falls may include at least one of determining a plurality of Voronoi cells based on the plurality of first location parameters as seeds, building a kd tree based on the plurality of first location parameters, and dividing the parameter space into uniform segments.

[0077] The methods of the first and / or second aspect may be implemented in at least one graphics processing unit.

[0078] Each of the method steps may be implemented in at least one graphics processing unit. The graphics processing unit may include multiple processing cores. One or more aspects of the method steps may be implemented in parallel with one another on different processing cores. In some examples, one or more of the method steps may be implemented in multiple graphics processing units. For example, one or more of the method steps may be implemented in multiple graphics processing units operating in parallel with one another.

[0079] The compression and decompression methods disclosed herein are particularly suitable for parallelization. For example, the detection of a first subset of location parameters (performed as part of the compression method) that indicate the location of discontinuities in a measure of a scene characteristic is a task that can be easily parallelized to allow multiple cores and / or multiple processing units (e.g., graphics processing units) to perform parts of the detection in parallel. For example, different parts of the parameter space can be analyzed by different cores and / or processing units to identify different discontinuities in parallel.

[0080] Furthermore, determining a subset of the plurality of first location parameters indicative of first locations located near the second location indicated by the second location parameter (performed as part of the decompression method) is a task that can be easily parallelized such that multiple cores and / or multiple processing units (e.g., graphics processing units) can perform the subset determination in parallel, for example, different subsets associated with different second locations may be determined in parallel by different cores and / or processing units.

[0081] Furthermore, interpolating the first measures of the characteristic of the scene corresponding to a subset of the plurality of first parameters to determine the second measures of the characteristic of the scene corresponding to the second location parameters (performed as part of the decompression method) is a task that can be easily parallelized such that multiple cores and / or multiple processing units (e.g., graphics processing units) can perform the interpolation in parallel. For example, the second measures of the characteristic of the scene corresponding to different second location parameters may be determined in parallel by different cores and / or processing units.

[0082] According to a third aspect of the present disclosure, there is provided a method of distributing a representation of a three-dimensional scene, the method comprising: compressing the representation of the three-dimensional scene according to the method of the first aspect, performing the method by at least one first computing device, transmitting the compressed representation of the three-dimensional scene to at least one second computing device, and decompressing the compressed representation of the three-dimensional scene according to the method of the second aspect, performing the method by the at least one second computing device.

[0083] The method may further include rendering, at at least one second computing device, an image of the three-dimensional scene using the decompressed representation of the three-dimensional scene.

[0084] According to a fourth aspect of the present disclosure, there is provided a computer-implemented method of rendering an image for display, the method comprising decompressing a compressed representation of a three-dimensional scene according to the method of the second aspect, and rendering an image of the three-dimensional scene using the decompressed representation of the three-dimensional scene.

[0085] According to a fifth aspect of the present disclosure, there is provided a computer-implemented method for compressing a depth map representing a surface in a three-dimensional scene, the depth map including a plurality of distances from a viewing position to the surface in the three-dimensional scene, the plurality of distances corresponding to different viewing directions from the viewing position, the method including: detecting a first subset of viewing directions representing viewing directions for which a discontinuity occurs in the plurality of distances corresponding to the different viewing directions, and storing the first subset of viewing directions and the distances associated with the first subset of viewing directions as a compressed depth map.

[0086] According to a sixth aspect of the present disclosure, there is provided a computer-implemented method of decompressing a compressed depth map representing a surface in a three-dimensional scene. The compressed depth map includes a first plurality of distances from a viewing position to a surface in the three-dimensional scene along a first plurality of viewing position directions from the viewing position. The method includes determining a second plurality of distances based on the first plurality of distances, the second plurality of distances being distances from the viewing position to the surface in the three-dimensional scene along the second plurality of viewing position directions, the determining the second plurality of distances including, for each viewing direction of the second plurality of viewing directions, i) determining a subset of the first plurality of viewing directions that are close to the viewing direction of the second plurality of viewing directions, and ii) interpolating the first plurality of distances along the subset of the first plurality of viewing directions to determine a distance to the surface in the three-dimensional scene along the viewing position direction of the second plurality of viewing directions. The method further includes forming a decompressed depth map, the decompressed depth map including the first plurality of distances and the determined second plurality of distances.

[0087] According to a seventh aspect of the present disclosure, there is provided a computing device configured to compress a representation of a three-dimensional scene, the representation including a measure of a property of the scene corresponding to a plurality of different location parameters indicative of a location within the scene. The computing device comprises at least one processing unit and a memory storing instructions that, when executed by one or more processors, cause the device to: i) detect a first subset of the plurality of location parameters indicative of a location of a discontinuity in the measure of the property of the scene; and ii) store the first subset of the plurality of location parameters and the measure of the property of the scene corresponding to the first subset of the plurality of location parameters as a compressed representation of the scene.

[0088] The at least one processing unit may include at least one graphics processing unit. The instructions may be configured to be executed by the at least one graphics processing unit.

[0089] According to an eighth aspect of the present disclosure, there is provided a computer device configured to decompress a compressed representation of a three-dimensional scene, the compressed representation including a first measure of a characteristic of the scene corresponding to a plurality of first location parameters indicative of a first location within the scene. The computer device includes at least one processing unit and a memory storing instructions, which when executed by one or more processors, cause the device to i) determine a second measure of a characteristic of the scene corresponding to a plurality of second location parameters indicative of a second location within the scene, the determining the second measure of a characteristic of the scene corresponding to the plurality of second location parameters including: for each second location parameter, determining a subset of the plurality of first location parameters indicative of a first location located near the second location indicated by the second location parameter; and interpolating the first measure of a characteristic of the scene corresponding to the determined subset of the plurality of first parameters to determine the second measure of a characteristic of the scene corresponding to the second location parameter. The instructions further cause ii) to form a decompressed representation of the three-dimensional scene, the decompressed representation including the first measure of a characteristic of the scene corresponding to the plurality of first location parameters and the determined second measure of a characteristic of the scene corresponding to the plurality of second location parameters.

[0090] The at least one processing unit may include at least one graphics processing unit. The instructions may be configured to be executed by the at least one graphics processing unit.

[0091] According to a ninth aspect of the present disclosure, there is provided a computing device configured to compress a depth map representing a surface in a three-dimensional scene, the depth map including a plurality of distances from a viewing position in the three-dimensional scene to the surface, the plurality of distances corresponding to different viewing directions from the viewing position. The computing device includes at least one processing unit and a memory storing instructions that, when executed by one or more processors, cause the device to: i) detect a first subset of viewing directions representing viewing directions for which a discontinuity occurs in the plurality of distances corresponding to the different viewing directions; and ii) store the first subset of viewing directions and the distances associated with the first subset of viewing directions as a compressed depth map.

[0092] According to a tenth aspect of the present disclosure, there is provided a computer device configured to decompress a compressed depth map representing a surface in a three-dimensional scene. The compressed depth map includes a first plurality of distances from a viewing position to a surface in the three-dimensional scene along a first plurality of viewing directions from the viewing position. The computer device comprises at least one processing unit and a memory storing instructions. The instructions, when executed by one or more processors, cause the device to determine a second plurality of distances based on the first plurality of distances, the second plurality of distances being distances from the viewing position to the surface in the three-dimensional scene along the second plurality of viewing position directions. Determining the second plurality of distances includes, for each viewing direction of the second plurality of viewing directions, determining a subset of the first plurality of viewing directions that are proximate to the viewing direction of the second plurality of viewing directions, and interpolating the first plurality of distances along the subset of the first plurality of viewing directions to determine distances to the surface in the three-dimensional scene along the viewing direction of the second plurality of viewing directions. The instructions further cause forming a decompressed depth map, the decompressed depth map including the first plurality of distances and the determined second plurality of distances.

[0093] It is expressly intended that within the scope of this application, the various aspects, embodiments, examples, and alternatives presented in the preceding paragraphs, claims, and / or in the following description and drawings, in particular their individual features, may be employed independently or in any combination. That is, the features of all examples and / or any examples may be combined in any manner and / or combination, so long as such features are not inconsistent. Applicants reserve the right to modify the originally filed claims or to submit new claims accordingly, including the right to amend the originally filed claims to depend on or incorporate features of any other claims, even if not originally so claimed. [Brief description of the drawings]

[0094] One or more embodiments of the invention are illustrated diagrammatically, for purposes of example only, in the accompanying drawings. [Figure 1] 1 is a schematic diagram of a system for generating, rendering and displaying images of a three-dimensional scene. [Diagram 2] 1 is a schematic diagram of a depth map of an example three-dimensional scene. [Diagram 3] 1 is a flowchart of a method for compressing a representation of a three-dimensional scene. [Figure 4] FIG. 3 is a schematic diagram of the determined second derivative of the depth map of FIG. 2; [Diagram 5] 1 is a flowchart of a method for decompressing a compressed representation of a three-dimensional scene. [Figure 6] FIG. 2 is a schematic diagram showing a parameter space divided into segments and a number of first positions within the parameter space; [Figure 7] FIG. 2 is a schematic diagram of a parameter space divided into Voronoi cells and a number of first positions in the parameter space; [Figure 8] 1 is a schematic diagram of a parameter space divided into a number of segments based on a target second position and a number of first positions in the parameter space; [Figure 9]FIG. 2 is a schematic diagram of a subset of first locations used to determine a measure of a property of the three-dimensional scene for a second location; [Figure 10] 3 is a schematic diagram of a portion of the depth map of FIG. 2 in its original uncompressed form and after being compressed and decompressed using two different methods; [Figure 11] 1 is a schematic diagram of a computing device that can be used to implement aspects of one or more of the methods described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0095] Before describing specific embodiments of the present invention, it should be understood that the present disclosure is not limited to the specific embodiments described herein, and that the terminology used herein is used only to describe specific embodiments, and is not intended to limit the scope of the claims.

[0096] In describing and claiming the computer apparatus and methods of the present invention, the following terminology is used: the singular forms "a," "an," and "the" include the plurals unless the context clearly dictates otherwise. Thus, for example, reference to a "processing unit" includes reference to one or more of such elements.

[0097] FIG. 1 is a schematic diagram of a system 100 for generating, rendering, and displaying an image of a three-dimensional scene. The system 100 includes a media server 101, a rendering module 102, and a display device 103. The media server 101 can generate, receive, or store information characterizing a three-dimensional scene to be displayed by the display device 103. In at least some examples, the three-dimensional scene may include a virtual component for display, for example, as part of an extended reality (XR) experience. The three-dimensional scene may be represented by a number of components that allow the scene to be rendered for display. For example, a representation of a three-dimensional scene may include components that characterize properties such as geometry, material properties, viewing position, textures, lighting, shading information, etc., that completely describe the scene to be displayed with sufficient detail such that an image of the scene can be rendered. This information may be generated at, received at, and / or stored at the media server 101.

[0098] A representation of a three-dimensional scene for use in rendering may include measures of scene properties corresponding to a number of different location parameters indicative of a location within the scene. That is, a particular property of a scene may be expressed for a number of different locations within the scene. The properties of the scene may include properties related to surfaces within the scene. For example, the properties may be indicative of geometric properties of surfaces within the scene, such as the location, shape and / or orientation (e.g., surface normal direction). Additionally or alternatively, the properties may be indicative of surface properties, such as the material or optical properties of the surface. For example, properties such as surface reflectance (which may be represented as one or more colors of albedo), distortion, and / or color may be represented as a function of location within the scene. Additionally or alternatively, one or more properties indicative of lighting, illumination, and / or shadows within the scene may be represented as a function of location within the scene. In general, properties that may be used to render an image of a scene may be represented as measures of scene properties corresponding to a number of different location parameters indicative of a location within the scene.

[0099] The different location parameters, which indicate different locations within a scene, may each include one or more coordinates that define a location in one or more dimensions. For example, each different location parameter may include a set of coordinates that define a location within a scene in two or three dimensions. In such an example, a measure of a property of the scene may be expressed for the different two or three dimensional locations within the scene.

[0100] A two-dimensional location in the three-dimensional scene may be defined as an x-axis location and a y-axis location. The x-axis location and the y-axis location may correspond to a pixel location in an image that is rendered based on the representation of the three-dimensional scene. Equivalently, the x-axis location and the y-axis location may correspond to two viewing angles or directions, respectively, in two dimensions from a viewing position. The viewing position may represent an observation position relative to which an image is rendered based on the representation of the three-dimensional scene. In at least some examples, the characteristics of the scene may include a three-dimensional location represented by a number of different position parameters that define the location in the two dimensions.

[0101] To aid in understanding the scene representations considered herein, an example of a representation in the form of a depth map is described and illustrated in detail, although it should be understood that the methods and functionality described herein may also be applied to other representation formats with different characteristics than those detailed herein.

[0102] A depth map can represent the distance from a viewing position to surfaces in a three-dimensional scene when viewing along different viewing directions. The different viewing directions can be thought of as different rays (characterized by straight lines) that each pass through the viewing position. For example, the different viewing directions may correspond to rays that are used to render the scene to form an image (e.g., when using a rendering technique such as ray casting).

[0103] The distances corresponding to different viewing directions are an example of a measure of a property (distance to a surface) corresponding to a number of different position parameters (different viewing directions). The distances corresponding to different viewing directions can be visualized as a grayscale image. The different viewing directions represented by the depth map can correspond to different pixels of a rendered image displayed by the display device 103. That is, the pixels of the depth map can directly correspond to pixels of an image rendered using the depth map.

[0104] FIG. 2 is a schematic diagram of a depth map 200 of an example three-dimensional scene. The example depth map is shown in FIG. 2 as a grayscale image. Each pixel of the grayscale image is displayed at a different location on the x-axis and y-axis labeled in FIG. 2 (similarly, the u-axis and v-axis can be used to indicate location in two dimensions). The different locations on the x-axis and y-axis represent different viewing directions relative to a reference viewing position. The grayscale intensity of each pixel represents the distance of a surface from the viewing position in the three-dimensional scene along the viewing direction to which the pixel corresponds. For example, surfaces closer to the viewing position can be displayed as a darker tone, and surfaces further away from the viewing position can be displayed as a lighter tone.

[0105] A depth map can be stored as a two-dimensional array of distances from a viewing position to surfaces in a three-dimensional scene. Different positions in the two-dimensional array can represent different viewing directions from the viewing position. For example, different columns in the two-dimensional array can correspond to different positions on an x-axis (e.g., the x-axis shown in FIG. 2). Different rows in the two-dimensional array can correspond to different positions on a y-axis (e.g., the y-axis shown in FIG. 2).

[0106] A depth map may define surface locations using the form of a three-dimensional coordinate system. The distance stored in the depth map as a measure of a feature (distance) represents one dimension of the coordinate system. The two dimensions that define where in the depth map the distance is stored (e.g. column number and row number, or equivalently x position and y position) represent the remaining two dimensions of the coordinate system. The two dimensions that define where in the depth map the distance is stored correspond to the two dimensions that define the viewing direction from the viewing position, and can be thought of as multiple different location parameters to which each distance corresponds.

[0107] In at least some examples, a depth map representing a surface in a three-dimensional scene may be generated from an alternative representation of the surface in the three-dimensional scene. For example, the surface in the three-dimensional scene may be represented by a collection of polygons that form the surface in the three-dimensional scene. The depth map may be generated (e.g., by the media server 101) for a defined viewing position, and the depth map may be generated by calculating distances from the viewing position to the surface in the three-dimensional scene along different viewing directions.

[0108] As described above, a representation of a property of a three-dimensional scene (e.g., a depth map, etc.) can be used to render an image of the scene for display. In at least some examples, multiple different representations, each associated with a different property, can be used to render the image of the scene. The different properties (e.g., relating to geometry and / or material / optical properties of surfaces) can each be represented as a measure of a given property corresponding to multiple different position parameters. The multiple different position parameters may be the same for one or more of the different properties. For example, the multiple different representations, each associated with the different properties, may include various measures of the property corresponding to the same range of viewing directions from a viewing position (or equivalently, a pixel position in the rendered image).

[0109] These representations of one or more characteristics of the scene corresponding to the different position parameters, along with further information characterizing the scene, can be provided from the media server 101 to the rendering module 102. The rendering module 102 uses the information provided by the media server 101 (including one or more representations of the scene, e.g., a depth map, etc.) to render an image of the three-dimensional scene for display by the display device 103. For example, the rendering module 102 may perform deferred rendering to generate an image for display. The rendered image for display may be represented by one or more luminance values ​​and corresponds to a number of pixels.

[0110] Display device 101 may be any suitable form of display, such as one or more two-dimensional displays, three-dimensional displays, and / or projectors.

[0111] The images rendered by rendering module 102 for display by display device 103 may represent frames of moving visual media (e.g., video, etc.). The moving visual media may be displayed by the display device at a frame rate of multiple frames per second (e.g., tens of frames per second, e.g., 60 frames / second). That is, display device 103 may display multiple different images (representing frames of the moving visual media) per second.

[0112] In at least some examples, the images displayed by the display device 103 may be rendered in real-time, i.e., the rendering module 102 may render the images at a frame rate that corresponds to the frame rate at which the frames are displayed by the display device 103 (e.g., tens of frames per second).

[0113] In at least some examples, the rendered and displayed three-dimensional scene may be generated in real time. For example, the media server 101 (or other module) may generate the rendered and displayed three-dimensional scene at a frequency corresponding to the frame rate at which frames are rendered and displayed by the rendering module 102 and the display device 103. This may be especially true if the rendered three-dimensional scene and / or the displayed images are responsive to some real-time input.

[0114] The functionality of the media server 101, the rendering module 102, and / or the display device 103 may be performed by different devices, as shown diagrammatically in the system 100 of FIG. 1. However, in some examples, at least some functionality of the media server 101, the rendering module 102, and / or the display device may be implemented by a single device. That is, some aspects of the media server 101, the rendering module 102, and / or the display device 103 may be implemented in a single device. In some examples, the functionality of the media server 101, the rendering module 102, and / or the display device 103 may be distributed across multiple devices. For example, the rendering module 102 may comprise a distributed rendering system including multiple devices connected through a network. Different devices in the distributed rendering system may perform aspects of a rendering task in parallel (e.g., the distributed devices may render different portions of a frame in parallel). A distributed rendering system may provide improved rendering speed (relative to rendering performed by a single device), which may be particularly important when performing real-time rendering.

[0115] It will be appreciated that when implementing system 100 for generating, rendering, and displaying images of a three-dimensional scene using multiple devices, one or more representations of the scene may need to be communicated between the devices, for example, over a network. For example, at least one representation of a characteristic of the scene (e.g., a depth map, etc.) may be generated and / or stored at media server 101 and transmitted to rendering module 102 for use in rendering the image. Additionally or alternatively, the representation of the characteristic of the scene may be communicated between different devices forming a distributed rendering system.

[0116] The transmission of one or more representations of a scene (e.g., depth maps, etc.) over a network between devices can use a significant amount of bandwidth. Taking depth maps as an example, it may be desirable to maintain a relatively large bit depth to represent the distances in the depth map in order to maintain sufficient accuracy in rendering based on the depth map. For example, each distance can be stored with a bit depth of 24 or 32 bits. For example, when rendering an image at 4k or ultra-high definition (UHD) resolution, a single depth map may represent up to about 32 megabytes worth of data. For real-time rendering applications, it may be necessary to transmit tens of depth maps per second over the network, corresponding to a video with tens of frames per second (e.g., 60 frames per second). At a frame rate of 60 frames per second, transmitting a depth map corresponding to a 4k or UHD resolution can require about 2 gigabytes per second of network bandwidth just for the transmission of the depth map. Furthermore, modern distributed rendering tasks are reaching resolutions of 8k, 12k and above, further increasing the network bandwidth required for real-time rendering applications.

[0117] Although an example bandwidth has been provided for the transmission of a depth map in a real-time rendering application, as further described above, one or more further representations of the characteristics of the three-dimensional scene may also be used for rendering the image. Such representations may each include measures of the characteristics of the scene for a similar number of position parameters (pixels) as the depth map, and may also require transmission at a corresponding frame rate. Thus, such further representations of the characteristics of the three-dimensional scene also require a significant amount of network bandwidth for a real-time rendering application.

[0118] To reduce network bandwidth requirements, it is often desirable to compress a representation of a three-dimensional scene (e.g., a depth map, etc.) to generate a compressed representation that has a smaller memory footprint than the original uncompressed representation. The compressed representation of the scene can be transmitted over a network using less network bandwidth than transmitting the original uncompressed representation. For example, in the system of FIG. 1, the media server 101 can compress a representation of a three-dimensional scene (e.g., a depth map, etc.) and transmit the compressed representation to the rendering module 102. The rendering module 102 can decompress the compressed representation to generate a decompressed representation that replicates the original uncompressed representation as closely as possible. The rendering module 102 can use the decompressed representation to render an image of the scene.

[0119] The advantages of compressing and decompressing a representation of a three-dimensional scene have been discussed above in the context of a real-time rendering application in terms of reducing the network bandwidth required to transmit the representation of the scene. However, compressing and decompressing a representation of a scene may have further advantages beyond just reducing network bandwidth and not limited to real-time rendering applications. For example, compression of a representation of a scene may be used simply to reduce the memory footprint required to store the representation. For example, a representation of a scene may be pre-generated and stored in memory until use. In such a case, compression of the representation prior to storage may reduce the memory footprint required to store the representation.

[0120] Possible methods for compressing the representation of the three-dimensional scene include known image compression methods. For example, the representation of the three-dimensional scene can be compressed using image compression used in video encoding methods (e.g., video encoding standards such as the H.264 standard). However, it has been found that common image compression methods are often not suitable for compressing the representation of the three-dimensional scene.

[0121] In particular, it has been recognized that the information contained in the representation of a scene characteristic can often be characterized by low frequency information, and high frequency information can be discarded. For example, the scene characteristic may be relatively constant as a function of position at the edge of an object, often subject to abrupt transitions (discontinuities), and / or have a relatively constant rate of change at locations between transitions. Taking the depth map shown in FIG. 2 as an example, it can be seen that the depth represented in the depth map is subject to several discontinuities that may represent transitions between objects, or different surfaces of an object. Many image compression methods (e.g., such as those commonly used in video encoding standards) utilize macroblock encoding. It has been found that such macroblock encoding often degrades discontinuities in the compression process, which can lead to loss of information.

[0122] The methods described herein have proven particularly advantageous for compressing representations of three-dimensional scenes, such as depth maps. For example, the compression and decompression methods described herein may be able to achieve relatively high compression ratios with relatively low information loss. Furthermore, the compression and decompression methods described herein are particularly suitable for parallelization, such that multiple processing cores and / or processing units can execute parts of the compression and decompression methods in parallel. This allows the methods to be implemented entirely on one or more graphic processing units (GPUs), allowing for full GPU parallelization. Thus, the methods described herein allow for compression and decompression speeds sufficient to realize real-time rendering applications.

[0123] 3 is a flow chart of a method for compressing a representation of a three-dimensional scene. The method may be implemented in any suitable computing device or apparatus. For example, the method may be implemented by the media server 101 of FIG. 1. In at least some examples, the method may be implemented using a graphics processing unit (GPU).

[0124] As mentioned above, a representation of a three-dimensional scene may include measures of scene properties corresponding to a number of different location parameters indicative of a location in the scene. The different location parameters indicate different locations in a parameter space defined by the location parameters. For example, the location parameters may relate to pixel locations in a rendered image, which may be considered equivalent to a viewing direction from a viewing position relative to which the image is rendered. An example of a representation used to explain the method of FIG. 3 is a depth map, where the measures of scene properties include distances from a viewing position to surfaces in the scene, and the number of different location parameters include different viewing directions from the viewing position (or equivalently pixel locations in an image rendered based on the depth map). An example of a depth map to which the method of FIG. 3 can be applied is depicted diagrammatically in FIG. 2, as mentioned above. The method described here may also be applied to representations of other properties of a three-dimensional scene.

[0125] In step 301 of FIG. 3, a first subset of a plurality of location parameters is detected. The detected first subset of a plurality of parameters indicates locations of discontinuities in a measure of a characteristic of the scene. The detected first subset of a plurality of location parameters may represent locations where the measure of the characteristic of the scene undergoes a discontinuity (e.g., an abrupt transition, such as a step change) with a change in the location parameters (or equivalently, a change in position in a parameter space defined by the location parameters). For example, in examples where the location parameters represent a viewing direction from a viewing position and / or a pixel position in an image, the discontinuity locations may represent locations where a discontinuity occurs in the measure of the characteristic of the scene between adjacent viewing directions and / or pixel positions.

[0126] The discontinuities in the scale of the scene characteristics may be detected using any suitable technique, for example, one or more edge detection techniques used in image processing applications may be used to identify discontinuities in the scale of the scene characteristics.

[0127] In at least some examples, a measure of the derivative of the measure of the characteristic of the scene with respect to a change in the position parameter can be determined. For example, a measure of the first and / or second derivative of the measure of the characteristic of the scene with respect to a change in the position parameter can be determined. Taking the example of a depth map (such as the depth map shown in FIG. 2), a derivative of the distance to a surface represented by the depth map can be determined as a function of a change in the viewing direction (or equivalently, pixel position).

[0128] FIG. 4 is a schematic diagram 400 of the second derivative of the depth map shown in FIG. 2, plotted on the same x- and y-axes used in FIG. 2. In the schematic diagram of FIG. 4, lighter tones are used to indicate larger values ​​of the absolute value (magnitude) of the second derivative, and darker tones are used to indicate smaller values ​​of the absolute value of the second derivative. The calculated second derivative includes a combination of the second derivative with respect to the change in the x-direction and the second derivative with respect to the change in the y-direction. In the particular example of FIG. 2, the second derivative is determined by applying a Laplacian operator to the distances in the depth map as a function of the change in x and y positions.

[0129] In at least some examples, the derivative (e.g., the first derivative or the second derivative) can be determined by performing a convolution of a kernel with a representation of the scene. For example, the second derivative shown in FIG. 4 is determined by performing a convolution of a Laplacian kernel with the depth map of FIG. 2. Any suitable kernel can be used to calculate the first or second derivative, such as a Sobel kernel, a Scharr kernel, a Prewitt kernel, and / or a kernel that calculates the difference of Gaussians.

[0130] As can be seen in Figure 4, the absolute value of the second derivative of the depth map of Figure 2 contains discrete regions where the absolute second derivative is relatively high (the white lines appearing in Figure 4) and large regions where the absolute second derivative is relatively low (the dark regions between the white lines in Figure 4). The discrete regions of relatively high second derivative represent locations of discontinuities in the distance represented by the depth map (as can be seen by comparing Figures 2 and 4).

[0131] In at least some examples, a first subset of the plurality of location parameters can be determined as location parameters for which the determined magnitude of the derivative is greater than a derivative threshold. For example, locations within the image shown in FIG. 4 for which the second derivative is greater than a second derivative threshold can be determined. That is, pixel locations on the x-axis and y-axis (or equivalently, viewing direction from the viewing position) can be determined where the second derivative is greater than a threshold. The location parameters corresponding to the locations for which the determined magnitude of the derivative is greater than the derivative threshold are determined as a first subset of location parameters indicative of locations of discontinuities in the scale of the characteristic of the scene.

[0132] The derivative threshold may be a predefined threshold and may remain constant, for example, for processing a number of different scenes. Alternatively, the derivative threshold may be determined depending on the scene representation being processed. For example, the derivative threshold may be determined depending on the number of position parameters whose determined derivative magnitude is greater than the derivative threshold. For example, it may be desirable to achieve a target compression ratio, for example 10:1. The derivative threshold may be determined, for example, such that the number of position parameters whose determined derivative magnitude is greater than the derivative threshold is less than or equal to 10% of the total position parameters to achieve the target compression ratio. In other examples, different target compression ratios may be used to determine the derivative threshold.

[0133] In step 302 of Figure 3, a first subset of the plurality of position parameters (determined in step 301) and a measure of a characteristic of the scene corresponding to the first subset of position parameters are stored as a compressed representation of the scene. For example, in the case of a depth map, the compressed representation of the depth map includes pixel locations (or equivalently viewing directions) at which discontinuities were detected (first subset of position parameters detected in step 301) at distances forming the depth map, and distances corresponding to the locations of the discontinuities. The compressed representation of the scene may be considered to be a portion of the original representation, the portion being selected to correspond to the discontinuities in the characteristic of the scene.

[0134] In at least some examples, the compressed representation may include only the first subset of location parameters and measures of scene characteristics corresponding to the first subset of location parameters. That is, the data points corresponding to discontinuities in the scene characteristics may be the only data included in the compressed representation. In some examples, additional metadata may be included in the compressed representation to aid in interpretation and decompression of the compressed representation.

[0135] In some examples, one or more additional data points (in addition to those found in step 301) may be included in the compressed representation. For example, the method of FIG. 3 may further include sampling a plurality of different location parameters to determine a second subset of the plurality of location parameters (in addition to the first subset found in step 301). The sampling may include any suitable sampling method, such as, for example, a random sampling method. Additionally or alternatively, the location parameters may be sampled at uniform intervals. For example, every nth pixel (where n is an integer) of the depth map (or other equivalent representation) in the x and / or y directions may be sampled to determine the second subset of the plurality of location parameters. In some examples, regions of parameter space between location parameters in the first subset of location parameters may be sampled.

[0136] A second subset of the plurality of location parameters (determined by the sampling technique) and a measure of the characteristic of the scene corresponding to the second subset of location parameters may be included in the compressed representation of the scene, i.e., the compressed representation may include the first subset of location parameters, the measure of the characteristic of the scene corresponding to the first subset of location parameters, the second subset of location parameters, and the measure of the characteristic of the scene corresponding to the second subset of location parameters.

[0137] A second subset of the position parameters (and measures of the scene characteristics corresponding to the second subset of the position parameters) may be included in the compressed representation to provide information about the characteristics of the scene in the parameter space region between the discrete positions. For example, sampling additional points included in the second subset can provide information about the nature (e.g., shape, gradient, etc.) of the variation of the scene characteristics between the discrete positions (first subset of the position parameters). Thus, including the second subset of the position parameters and corresponding measures of the scene characteristics in the compressed representation can improve the accuracy with which the original representation of the scene can be reproduced based on the compressed representation.

[0138] As mentioned above, in at least some examples, additional data points (in addition to the detected first subset of location parameters corresponding to the discontinuity) may be included in the compressed representation of the scene. For example, a second subset of location parameters determined by sampling may also be included in the compressed representation. However, even in such examples, the compressed representation of the scene may be fully represented as a subset of the different location parameters included in the uncompressed representation and measures of the scene characteristics corresponding to the subset of the different location parameters. The subset of the different location parameters is at least included in the first subset detected in step 301. In at least some examples, the subset of the different location parameters representing the compressed representation may include one or more additional subsets of location parameters (such as, for example, the second subset described above) and measures of the scene characteristics corresponding to the one or more additional subsets.

[0139] The compressed representation of a scene may not include additional information defining how a measure of a characteristic of the scene varies among the positions indicated by a subset of position parameters included in the compressed representation. For example, the method of compressing the representation may not include fitting a function to the measure of the characteristic of the scene. The compressed representation may not include function parameters that define a function that represents the variation of the measure of the characteristic of the scene.

[0140] The compression method of a scene representation described above with reference to Fig. 3 may be particularly suitable for parallelization. That is, different parts of one or more steps of the compression method may be performed in parallel by different processing cores and / or processing units. The compression method is particularly suitable for parallelization since related computations can often be performed independently of other computations that form part of the same method step. For example, different processing cores and / or processing units may determine derivatives of measures of scene properties in parallel and independently of each other for different parts of the scene. That is, at least some aspects of step 301 of Fig. 3 may be performed in parallel by different processing cores and / or processing units for different parts of the scene. The suitability of the compression method for parallelization may mean that certain advantages can be realized by implementing the compression method on one or more GPUs.

[0141] As described above, a representation of a scene (e.g., a depth map, etc.) can be compressed at a first computing device or apparatus (e.g., such as the media server 101 of FIG. 1 ) and the compressed representation can be transmitted to a second computing device or apparatus (e.g., such as the rendering module 102). The compressed representation can be decompressed (e.g., at the second computing device or apparatus, such as the rendering module 102) to recreate the original uncompressed representation from the compressed representation. Decompressing a compressed representation is particularly useful, finding particular application when the representation is used to render an image of the scene, because such applications may require measures of characteristics of the scene for each pixel of the rendered image.

[0142] 5 is a flow chart illustrating an example method for decompressing a compressed representation of a scene. The representation of the scene and the compressed representation of the scene may include any of the characteristics described above with respect to the methods for compressing the representation. For example, the compressed representation of the scene may have been compressed according to any of the compression methods and functions described above.

[0143] The compressed representation of the scene may include a first measure of a characteristic of the scene corresponding to a plurality of first location parameters indicative of a first location in the scene. The decompression method includes determining a second measure of a characteristic of the scene corresponding to a plurality of second location parameters indicative of a second location in the scene. For example, the first location parameters may correspond to a first subset of pixel locations in the image to be rendered (e.g. a subset of pixels or viewing directions of a depth map). The second location parameters may correspond to the remaining pixel locations (or equivalently the viewing directions) in the image. Thus, the decompression method is equivalent to recreating data points in the original scene representation that are missing from the compressed representation. The second measure of a characteristic of the scene corresponding to the plurality of second location parameters indicative of a second location in the scene (data points that are missing in the compressed representation) may be determined in response to the first measure of a characteristic of the scene corresponding to the plurality of first location parameters indicative of the first location in the scene (data points included in the compressed representation).

[0144] In step 501 of Figure 5, a target second location parameter indicative of a target second location within the scene is determined. The target second location parameter may be any second location parameter for which a second measure of a characteristic of the scene has not yet been determined. For example, the decompression method may include repeating the steps of the method of Figure 5 for each second location within the scene (i.e., for each data point missing from the compressed representation).

[0145] In step 502, a subset of first location parameters indicative of a first location that is proximate to the target second location is determined. The subset of first location parameters is determined based on the target second location. The subset of first location parameters may comprise, for example, a set of nearest neighbors to the target second location, or an approximation of the nearest neighbors. The subset of first location parameters may comprise only a portion of the nearest neighbors to the target second location.

[0146] A number of different methods can be used to determine the subset of the first position parameters. As a first step, the first position parameters can be sorted into an organized, systematic data structure to facilitate identification of the subset of the first position parameters. For example, the parameter space characterized by the first position parameters can be divided into a number of segments. The first position parameters can then be sorted into segments, and one or more segments that are proximate to the second position parameters are searched to identify the subset of the first position parameters.

[0147] Figure 6 is a schematic diagram of a parameter space on an x-axis and a y-axis and a number of first positions 601 in the parameter space. The parameter space may correspond, for example, to the positions of pixels of a rendered image. For example, the x-axis and y-axis shown in Figure 6 may correspond to the axes of Figures 2 and 4. Each of the first positions 601 marked with a dot in Figure 6 is defined by a first position parameter included in the compressed representation.

[0148] In the example of FIG. 6, the parameter space is divided into uniform segments 602, which are delimited by dashed lines as shown in FIG. 6. Each of the first locations 601 falls within one of the segments 602. FIG. 6 also shows an example of a target second location 603 located in a segment designated by reference numeral 602a in FIG. 6. In at least some examples, the subset of first location parameters can be determined simply as the first location parameters that define a location within the same segment as the target second location. For example, for the target second location 603 shown in FIG. 6, the subset of first location parameters can be determined as the first location parameters that define the first location 601 located in the same segment 602a as the target second location. In at least some examples, the subset of first location parameters can include one or more first locations located in segments adjacent to the segment 602a in which the target second location 603 is located.

[0149] In some examples, more advanced methods of dividing the parameter space into segments can be used. For example, the first location can be sorted onto a k-dimensional (kd) tree to divide the parameter space into segments. In some examples, the first location can be used as a seed for determining multiple Voronoi cells.

[0150] 7 is a schematic diagram of a number of Voronoi cells 702 determined for a number of first locations 601. The same xy parameter space is used in FIG. 7 as described above in relation to FIGS. 2, 4 and 6. Each first location 601 has a corresponding Voronoi cell 702, which represents the region of parameter space to which the respective first location 601 is closest to. For example, the first location labeled 601a in FIG. 7 is the first location 601 closest to all of the points that lie within the corresponding Voronoi cell labeled 702a in FIG. 7. The Voronoi cells can be determined using any suitable method, for example by using a jump flooding algorithm.

[0151] In an example where the parameter space is divided into segments by determining Voronoi cells 702 using multiple first locations 601 as seeds, the target second location can be evaluated to determine in which Voronoi cell 702 it is located. A subset of proximate first location parameters can then be determined to include the first location parameter of the first location corresponding to the Voronoi cell 702 in which the target second location is located and the first location parameters of one or more first locations corresponding to one or more adjacent Voronoi cells 702.

[0152] In step 503 of the method of Figure 5, a first measure of a characteristic of the scene corresponding to the determined subset of the plurality of first location parameters may be interpolated to determine a second measure of the characteristic of the scene corresponding to a target second location parameter. Any suitable interpolation method may be used, for example linear interpolation or inverse distance weighted interpolation. The method of determining the subset of the plurality of first location parameters in step 502 may depend on the interpolation method used in step 503. Simple examples of possible combinations of methods for use in steps 502, 503 are described with reference to Figures 8 and 9.

[0153] FIG. 8 is a schematic diagram of a number of first locations 801 and a target second location 803. FIG. 8 uses the same xy parameter space as described above in connection with FIG. 2, FIG. 4, FIG. 6, and FIG. 7. The parameter space shown in FIG. 8 is divided into segments 802a, 802b, 802c, and 802d based on the location of the target second location 803. In particular, the parameter space is divided into four segments 802a, 802b, 802c, and 802d, so that the target second location is located at the intersection of the four segments 802a, 802b, 802c, and 802d. Then, each segment 802a, 802b, 802c, and 802d is searched to identify the first location 801 that is closest to the target second location 803 in each segment. For example, the first location labeled 801a in FIG. 8 is identified as the closest first location in the top left segment 802a, as shown in FIG. 8. The first location labeled 801b in Figure 8 is identified as the closest first location in the top right segment 802b as shown in Figure 8. The first location labeled 801c in Figure 8 is identified as the closest first location in the bottom left segment 802c as shown in Figure 8. The first location labeled 801d in Figure 8 is identified as the closest first location in the bottom right segment 802d as shown in Figure 8.

[0154] The closest first positions 801a, 801b, 801b, 801d may be identified as possible first positions for inclusion in a first subset of first positions for the target second position 803 to perform linear interpolation. In some examples, three of the four closest first positions 801a, 801b, 801b, 801d may be selected to form the first subset of first positions for the target second position 803. Three of the four closest first positions 801a, 801b, 801b, 801d may be selected to satisfy a set of criteria. The criteria may include, for example, a condition that the three closest first positions are not collinear with each other (i.e., they are not all on a straight line) and a condition that the target second position 803 is inside a triangle formed by the three closest first positions. 8, the closest first positions 801a, 801b, 801c in the top left segment 802a, the top right segment 802b, and the bottom left segment 802c may be selected to satisfy the criteria set. These first positions 801a, 801b, 801c thus form a first subset of first positions on which the interpolation of step 503 is based.

[0155] FIG. 9 is a schematic diagram of a selected subset of first locations 801a, 801b, 801c in three-dimensional space. The x- and y-axes shown in FIG. 9 correspond to the x- and y-axes in FIG. 8. Also shown in FIG. 9 is a z-axis. Alternatively, the u- and v-axes may be used to indicate locations in two dimensions, and the x-, y-, and z-axes may be used to indicate locations in three dimensions. In the example of FIG. 9, the representation of the scene comprises a depth map, and the measures of the property of the scene include distances to surfaces in the scene corresponding to each first location 801. In the illustration of FIG. 9, the location of a point on the z-axis corresponds to the distance (depth) to the surface corresponding to each of the first locations 801a, 801b, 801c. A second measure of the property of the scene (distance to surface) of the target second location 803 can be determined by interpolating the first measures (distances) corresponding to the selected subset of first locations 801a, 801b, 801c. In the example of FIG. 9, linear interpolation is used. In particular, a plane 805 is determined in which each of the subsets of first locations 801a, 801b, 801c lies. A second measure of the property of the scene (distance to the surface) is determined by determining the position on the z-axis of the plane at the x- and y-location corresponding to the target second location 803.

[0156] Although a particular method of linear interpolation has been described above with reference to Figures 8 and 9, in general, any suitable form of interpolation may be used in step 503. For example, an inverse distance weighted interpolation method may be used. In such an example, a weight may be determined for each of the subsets of first location parameters determined in step 502. In particular, each weight depends on a distance (e.g., Euclidean distance) between the respective first location and the target second location. The distance may correspond to a distance in a parameter space (e.g., in the illustrated example, in the xy plane) to which the first and second location parameters relate. The determined weight may be inversely proportional to the distance between the respective first location and the target second location. That is, a first location located further away from the target second location is assigned a lower weight than a first location located closer to the target second location. The determined weight may be used to determine a weighted average of a first measure of a characteristic of the scene (e.g., distance to a surface) corresponding to the subset of the plurality of first parameters. In such an example, the measures corresponding to the first locations that are closer to the target second location may contribute more to the weighted average than the measures corresponding to the first locations that are farther from the target second location. The weighted average of the first measures of the scene characteristic may form a determined second measure of the scene characteristic for the target second location.

[0157] As described above, steps 501, 502, and 503 of the method of Figure 5 may be performed for each second location for which a second measure of a characteristic of the scene needs to be determined in order to replicate the original uncompressed representation of the scene based on the compressed representation. For example, multiple iterations of the method of Figure 5 may be performed to determine a second measure for each of a plurality of second locations.

[0158] Advantageously, the method of Figure 5 can be performed for each target second location parameter independently of other target second location parameters. That is, each second location parameter can be considered separately and a second measure of the scene characteristic (e.g. distance to a surface) can be determined for a given second location parameter independently of determining the second measure of the scene characteristic for other second location parameters. The ability to independently determine second measures of the scene characteristic corresponding to different second location parameters means that the decompression method is well suited to parallelization.

[0159] In at least some examples, the steps of the method of Figure 5 may be performed in parallel for different second locations and second location parameters. For example, different processing cores and / or different processing units may perform the steps of the method of Figure 5 in parallel for different second locations and second location parameters. The suitability of the decompression method for parallelization means that certain advantages can be realized by implementing the decompression method on one or more GPUs.

[0160] A decompressed representation of the scene is formed to include both first measures of the characteristic of the scene corresponding to the plurality of first location parameters and determined second measures of the characteristic of the scene corresponding to each of the second location parameters. The decompressed representation of the scene includes a recreation of the original uncompressed representation of the scene prior to compression.

[0161] The decompressed representation of the scene may be suitable for use in rendering an image of the scene. For example, taking the system 100 of Figure 1, the rendering module 102 may decompress the compressed representation of the scene received from the media server 101. The decompressed representation of the scene is used by the rendering module to render an image of the scene for display on the display device 103.

[0162] As mentioned above, it has been found that the compression and decompression methods disclosed herein are capable of reproducing the original uncompressed scene representation with a high degree of accuracy while achieving a high compression ratio. Figure 10 is a schematic diagram of a portion of the depth map of Figure 2 in its original uncompressed form and after compression and decompression using various compression and decompression methods. The leftmost portion of Figure 10 (labeled 901) is a representation of a portion of the depth map in its original uncompressed form. The central portion of Figure 10 (labeled 902) is a representation of the same portion of the same depth map after compressing and decompressing the depth map using a compression method using a discrete cosine transform (DCT). The rightmost portion of Figure 10 (labeled 903) is a representation of the same portion of the same depth map after compressing and decompressing the depth map using the compression and decompression methods described herein. That is, the portion of the depth map 903 shown in the rightmost portion of Figure 10 has been compressed using a method according to the steps of Figure 3, and the compressed representation has been decompressed using a method according to the steps of Figure 5.

[0163] The compression used to form the portion of the depth map 902 shown in the center portion of FIG. 10 (using the DCT compression method) achieved a compression ratio of about 4:1. That is, the memory footprint of the original uncompressed depth map was about four times larger than the memory footprint of the compressed depth map. The compression used to form the depth map 903 shown in the rightmost portion of FIG. 10 (using the compression method described herein) achieved a compression ratio of about 17:1. That is, the memory footprint of the original uncompressed depth map was about 17 times larger than the memory footprint of the compressed depth map. Thus, the compression method described herein can achieve a significantly higher compression ratio than an equivalent compression using the DCT method. As previously mentioned, achieving a high compression ratio can significantly reduce the network bandwidth required to transmit the compressed representation between devices.

[0164] A close comparison of the central portion 902 of Figure 10 (a decompressed portion of a depth map that has been compressed using a DCT scheme) with the right-hand portion 903 of Figure 10 (a decompressed portion of a depth map that has been compressed using the methods described herein) shows that the right-hand portion 903 of Figure 10 is a more accurate reproduction of the original uncompressed depth map 901. In particular, it can be seen from the central portion 902 of Figure 10 that the use of a DCT compression scheme degrades the appearance of transitions between various surfaces in the depth map when compared to the right-hand portion 903 of Figure 10, which more accurately reproduces the transitions between surfaces.

[0165] As noted above, the compression and decompression methods described herein have been found to reproduce the original uncompressed scene representation with greater accuracy than other forms of compression and decompression methods (e.g., using DCT compression). Furthermore, the compression methods described herein can achieve higher compression ratios than other compression methods, thereby significantly reducing network bandwidth and / or storage requirements.

[0166] The compression and decompression methods described herein have also been shown to be particularly suitable for parallelization. Thus, aspects of the methods described herein can be performed in parallel by different processing cores and / or processing units. In particular, the compression and / or decompression methods described herein can be implemented on one or more GPUs. The suitability of the methods described herein for parallelization means that significant speedups can be achieved by implementation on one or more GPUs.

[0167] Although specific examples have been described herein with reference to compressing and decompressing depth maps, any aspect or feature of the compression and / or decompression methods described herein may be performed on any representation of a three-dimensional scene suitable for rendering an image of the scene. In general, the compression and decompression methods described herein can be used to compress and / or decompress any representation of a three-dimensional scene that includes measures of characteristics of the scene that correspond to a number of different position parameters indicative of positions within the scene.

[0168] Aspects, steps, and / or features of the methods described herein may be performed by one or more computing devices or computing apparatus. For example, as discussed above, the compression methods described herein may be performed by a media server 101, which is an example of a computing device or computing apparatus. Additionally, the decompression methods described herein may be performed by one or more rendering modules 102, which may be embodied as one or more computing devices and / or computing apparatus.

[0169] The computing device or apparatus may be implemented in any suitable form. Figure 11 is a schematic diagram of a computing device 1000 or apparatus that may be used to implement all or part of the methods described herein. For example, the device 1000 or apparatus may be used to implement all or part of the media server 101 and / or the rendering module 102.

[0170] The device 1000 may include at least one processing unit 1001, a memory 1002, and an input / output (I / O) interface 1000. The processing unit 1001 may include any suitable processor and / or combination of processors. For example, the processing unit 1001 may include one or more of a central processing unit (CPU) and a graphical processing unit (GPU). The memory 1002 may include volatile memory and / or non-volatile / persistent memory. The memory 1002 may be used to store various data, such as, for example, an operating system, instructions executed by the processing unit (e.g., in the form of software executed by the processing unit), configuration information associated with the device 1000, and / or data associated with a scene to be rendered. For example, the memory 1002 may be used to store a representation of one or more characteristics of a three-dimensional scene. In some examples, the memory 1002 may be used to store instructions for performing any of the methods disclosed herein.

[0171] At least the processing unit 1001 can be connected to an input / output (I / O) interface 1003. The I / O interface 100 facilitates communication with one or more other devices, network nodes or modules in the network. For example, the I / O interface 1003 is operable to transmit and / or receive communications to and from other devices in the network. In some examples, the I / O interface 1003 is operable to transmit and / or receive communications over an air interface. For example, the I / O interface 1003 can include a transmitter and / or receiver for transmitting and / or receiving wireless communications (e.g., radio frequency signals). In some examples, the I / O interface 1003 can include a transceiver configured to receive and transmit wireless communications (e.g., radio frequency signals). In some examples, the I / O interface 1003 is additionally or alternatively operable to communicate over one or more wired connections.

[0172] Optionally, device 1000 may further include a display. For example, device 1000 may include a display for displaying information to a user of device 1000. The display may comprise any suitable electronic display. The display may be coupled to at least processing unit 1001. Processing unit 1001 may generate a display signal that is sent to the display to cause the display to display information.

[0173] It will be understood that the embodiments of the present invention can be realized in the form of hardware, software, or a combination of hardware and software. Such software can be stored in the form of volatile or non-volatile storage, such as, for example, a storage device such as a ROM, whether erasable or rewritable, or in the form of memory, such as a RAM, memory chips, devices, integrated circuits, or on an optically or magnetically readable medium, such as a CD, DVD, magnetic disk, magnetic tape, etc. It will be understood that the storage device and storage medium are machine-readable storage embodiments suitable for storing a program or programs that, when executed, implement the embodiments of the present invention. Thus, the embodiments provide a program including code for implementing the system or method claimed in any of the preceding claims, and a machine-readable storage for storing such a program. Furthermore, the embodiments of the present invention can be transmitted electronically over any medium, such as a communication signal carried over a wired or wireless connection, and the embodiments encompass it as appropriate.

[0174] It should be understood that features, integers, properties or groups described in connection with a particular aspect, embodiment or example of the invention are applicable to other aspects, embodiments or examples described herein, unless inconsistent therewith. All of the features disclosed in this specification (including the accompanying claims, abstract and drawings), and / or all of the steps of a method or process similarly disclosed, may be combined in any combination, except combinations in which at least some of such features and / or steps are mutually exclusive. The invention is not limited to the details of the foregoing examples.

Claims

1. A computer implementation method for compressing a representation of a three-dimensional scene, wherein the representation includes a measure of the scene's characteristics corresponding to a plurality of different position parameters indicating a position within the scene, A step of detecting a first subset of multiple positional parameters that indicate the location of discontinuities in the characteristics of the scene, A method comprising the steps of saving a first subset of multiple position parameters and a measure of scene characteristics corresponding to the first subset of multiple position parameters as a compressed representation of the scene.

2. The computer implementation method according to claim 1, wherein the measure of the scene's characteristics includes a measure of the characteristics of a surface in the scene corresponding to a plurality of different position parameters.

3. The computer implementation method according to claim 2, wherein the scale of the surface characteristics in the scene includes a scale indicating the geometry of the surface in the scene corresponding to a plurality of different position parameters.

4. The computer implementation method according to claim 1, wherein multiple different position parameters correspond to multiple different observation directions originating from the observation position, and each of the measures of scene characteristics corresponds to one of the multiple observation directions.

5. The representation of a 3D scene includes a depth map. The computer implementation method according to claim 4, wherein the measure of the scene characteristics includes the distance from the observation position to a surface in the three-dimensional scene along a plurality of different observation directions.

6. The step of detecting a first subset of multiple positional parameters that indicate the location of discontinuities in the characteristics of the scene is: The steps include determining the derivative of the measure of the scene's characteristics with respect to changes in the position parameter, The computer implementation method according to claim 1, comprising the step of determining a first subset of multiple position parameters as position parameters whose determined derivatives have a magnitude greater than a derivative threshold.

7. The computer implementation method according to claim 6, wherein the step of determining the derivative of the scale of the scene characteristics includes the step of determining the second derivative of the scale of the scene characteristics with respect to a change in the position parameter.

8. The computer implementation method according to claim 6 or 7, further comprising the step of determining a derivative threshold in accordance with the number of position parameters whose magnitude of the determined derivative is greater than the derivative threshold.

9. The steps include sampling multiple different position parameters to determine a second subset of the multiple position parameters, The computer implementation method according to claim 1, further comprising the step of storing, as a compressed representation of the scene, a first subset of multiple position parameters and a measure of the scene characteristics in the first subset of multiple position parameters, in addition to a second subset of multiple position parameters and a measure of the scene characteristics corresponding to the second subset of multiple position parameters.

10. A computer implementation method for decompressing a compressed representation of a three-dimensional scene, wherein the compressed representation includes a first measure of the scene's characteristics corresponding to a plurality of first position parameters indicating a first position within the scene, - A step of determining a second measure of the scene characteristics corresponding to a plurality of second position parameters indicating a second position within the scene, wherein the step of determining a second measure of the scene characteristics corresponding to a plurality of second position parameters is performed for each second position parameter, i) Determining a subset of first position parameters that indicate a first position located near a second position indicated by a second position parameter, ii) A step including interpolating a first measure of scene characteristics corresponding to a determined subset of multiple first parameters to determine a second measure of scene characteristics corresponding to a second position parameter, A computer implementation method comprising the steps of forming a decompressed representation of a three-dimensional scene, wherein the decompressed representation includes a first scale of scene characteristics corresponding to a plurality of first position parameters and a determined second scale of scene characteristics corresponding to a plurality of second position parameters.

11. The computer implementation method according to claim 10, wherein the first measure of the scene characteristics and the determined second measure each include measures of the surface characteristics in the scene corresponding to a plurality of first and second position parameters.

12. The computer implementation method according to claim 11, wherein the scale of the surface characteristics in the scene includes a scale indicating the geometric shape of the surface in the scene.

13. Multiple first position parameters and multiple second position parameters correspond to multiple different observation directions originating from the observation position. The computer implementation method according to any one of claims 10 to 12, wherein each of the first and second scales of the scene characteristics corresponds to one of a plurality of observation directions.

14. The compressed representation of a 3D scene includes a portion of the depth map. The computer implementation method according to claim 13, wherein the first and second scales of the scene characteristics include the distance to a surface in the three-dimensional scene along a plurality of different observation directions from the observation position.

15. The computer implementation method according to claim 10, wherein the interpolation of a first scale corresponding to a subset of multiple first parameters to determine a second scale of scene characteristics corresponding to a second position parameter includes linear interpolation.

16. Interpolating a first scale corresponding to a subset of multiple first parameters to determine a second scale of scene characteristics corresponding to a second position parameter is: - The weights associated with each subset of multiple first position parameters are determined according to the distance between the first position indicated by the first position parameter and the second position indicated by the second position parameter, The computer implementation method according to claim 10, comprising determining a weighted average of a first measure of scene characteristics corresponding to a subset of a plurality of first parameters, wherein the weighted average is determined using weights determined in association with each of the subsets of a plurality of first position parameters.

17. Determining a subset of first position parameters that indicate a first position located near a second position indicated by a second position parameter is: - Dividing the parameter space characterized by the first position parameter into multiple segments in which each of the first position parameters is contained, - Identify at least one adjacent segment that contains and / or is adjacent to the segment containing the second position parameter, The computer implementation method according to claim 10, comprising determining a subset of a plurality of first position parameters as at least a portion of a plurality of first position parameters located in at least one adjacent segment.

18. Dividing the parameter space characterized by the first positional parameter into multiple segments, each containing one of the first positional parameters, is: - As a seed, multiple voronoi cells are determined based on multiple first position parameters, - Constructing a k-d tree based on multiple first position parameters, The computer implementation method according to claim 17, further comprising dividing the parameter space into uniform segments.

19. The computer implementation method according to claim 1 or 10, wherein the method is implemented on at least one graphics processing unit.

20. A method for distributing the representation of a 3D scene, The method according to claim 1, which is performed by at least one first computer device, comprising the steps of compressing a representation of a three-dimensional scene, The steps include transmitting a compressed representation of a 3D scene to at least one second computer device, A method comprising the steps of decompressing a compressed representation of a three-dimensional scene according to the method of claim 11, which is performed by at least one second computer device.

21. The method according to claim 20, further comprising the step of rendering an image of a three-dimensional scene using a decompressed representation of the three-dimensional scene in at least one second computer device.

22. A computer implementation method for rendering an image for display, The steps of decompressing a compressed representation of a three-dimensional scene according to the method of claim 11, A method comprising the step of rendering an image of a 3D scene using a decompression representation of the 3D scene.

23. A computer implementation method for compressing a depth map representing a surface in a three-dimensional scene, wherein the depth map includes multiple distances from an observation position in the three-dimensional scene to the surface, and these multiple distances correspond to different observation directions from the observation position. A step of detecting a first subset of observation directions that represent observation directions in which discontinuities occur at multiple distances corresponding to different observation directions, A computer implementation method comprising the steps of saving a first subset of observation directions and the distances associated with the first subset of observation directions as a compressed depth map.

24. A computer implementation method for decompressing a compressed depth map representing a surface in a three-dimensional scene, wherein the compressed depth map includes a first plurality of distances from the observation position to the surface in the three-dimensional scene along a first plurality of observation directions from the observation position, - A step of determining a second set of distances based on a first set of distances, wherein the second set of distances is the distance from the observation position to the surface in the 3D scene along the second set of observation directions, and the step of determining the second set of distances is performed for each of the second set of observation directions, i) Determine a subset of the first set of observation directions that are close to the second set of observation directions, ii) A step comprising interpolating a first set of distances along a subset of first observation directions to determine the distance to a surface in a three-dimensional scene along one of the second set of observation directions, A computer implementation method comprising the steps of: forming a decompressed depth map, wherein the decompressed depth map includes a first plurality of distances and a determined second plurality of distances.

25. A computer device configured to compress a representation of a three-dimensional scene, wherein the representation includes a measure of the scene's characteristics corresponding to a plurality of different position parameters indicating a position within the scene, At least one processing unit, It has memory for storing instructions, If the instruction is executed by one or more processors, the device will, i) Detect a first subset of multiple positional parameters that indicate the location of discontinuities in the characteristics of the scene, and, ii) A computer device that stores a first subset of multiple position parameters and a measure of the scene characteristics corresponding to the first subset of multiple position parameters as a compressed representation of the scene.

26. A computer device configured to decompress a compressed representation of a three-dimensional scene, wherein the compressed representation includes a first measure of the scene's characteristics corresponding to a plurality of first position parameters indicating a first position within the scene, At least one processing unit, It has memory for storing instructions, If the instruction is executed by one or more processors, the device will, i) This involves determining a second measure of the scene's characteristics corresponding to multiple second position parameters that indicate a second position within the scene. Determining a second measure of the scene's characteristics corresponding to multiple second position parameters involves, for each second position parameter, To determine a subset of first position parameters that indicate a first position located near a second position indicated by a second position parameter, This includes interpolating a first measure of the scene characteristics corresponding to a determined subset of multiple first parameters to determine a second measure of the scene characteristics corresponding to a second position parameter, and, ii) A computer device that forms a decompressed representation of a three-dimensional scene, wherein the decompressed representation includes a first scale of scene characteristics corresponding to a plurality of first position parameters and a determined second scale of scene characteristics corresponding to a plurality of second position parameters.

27. A computer device configured to compress a depth map representing a surface in a three-dimensional scene, wherein the depth map includes multiple distances from an observation position in the three-dimensional scene to the surface, and these multiple distances correspond to different observation directions from the observation position. At least one processing unit, It has memory for storing instructions, If the instruction is executed by one or more processors, the device will, i) Detect a first subset of observation directions that represent observation directions in which discontinuities occur at multiple distances corresponding to different observation directions. ii) A computer device that stores a first subset of observation directions and the distances associated with the first subset of observation directions as a compressed depth map.

28. A computer device configured to decompress a compressed depth map representing a surface in a three-dimensional scene, wherein the compressed depth map includes a first plurality of distances from the observation position to the surface in the three-dimensional scene along a first plurality of observation directions from the observation position, At least one processing unit, It has memory for storing instructions, If the instruction is executed by one or more processors, the device will, i) Based on a first set of distances, a second set of distances is determined, and the second set of distances is the distance from the observation position to the surface in the 3D scene along the second set of observation directions, and determining the second set of distances is done for each viewing direction of the second set of observation directions, To determine a subset of the first set of observation directions that are close to one of the second set of observation directions, This includes interpolating a first set of distances along a subset of first observation directions to determine the distance to a surface in a three-dimensional scene along one of the second set of observation directions, and, ii) A computer device that forms a defrosting depth map, wherein the defrosting depth map includes a first plurality of distances and a determined second plurality of distances.

Images