Network-optimized, texture-assisted mesh compression

Abstract

A method is provided to perform network-optimized texture assisted mesh compression. The method includes receiving an original texture and an original mesh associated with an object. The method includes generating a plurality of compressed textures based on the original texture. For each compressed texture of the plurality of compressed textures, the method includes applying the machine learning model to a compressed texture and the original texture to generate a mesh estimate associated with the compressed texture, determining an error parameter between the mesh estimate associated with the compressed texture and the original mesh, and in response to the error parameter not exceeding a predefined threshold, determining an estimated network performance requirement. The method further includes selecting a selected compressed texture from the plurality of compressed textures and transmitting the selected compressed texture and the mesh estimate to a wireless communication device.

Description

NETWORK-OPTIMIZED, TEXTURE-ASSISTED MESH COMPRESSIONTECHNICAL FIELD

[0001] The present disclosure relates to a method and server for performing network- optimized, texture-assisted, mesh compression in a wireless communication system.BACKGROUND

[0002] Recent advancements in three-dimensional (3D) modeling technology have enabled the generation of geometric 3D meshes from texture information, leveraging deep learning techniques to create realistic and detailed models from various images. One method has focused on using segmented images alongside untextured mesh templates to generate textured 3D meshes through a process that includes pose estimation, silhouette matching, and the introduction of semantic information to resolve pose ambiguities. Other methods introduce deep learning architectures using similar inputs across several convolutional layers.

[0003] Another approach addresses the challenge of reconstructing 3D objects from multiview images, where the computationally expensive annotation of datasets has been a significant hurdle. By employing Neural Radiance Fields (NeRF) and Generative Adversarial Networks (GANs), this method aims to bridge the gap between synthetic and real images, focusing on generating high-quality texture maps that enhance the realism of the 3D models.

[0004] These techniques underscore the potential of combining neural networks, differentiable rendering, and GANs in creating textured 3D meshes. These techniques also have the potential to reduce the complexity of information rendered remotely in immersive experiences, which is what we explore in this application.SUMMARY

[0005] In an embodiment, a method to perform network-optimized texture-assisted mesh compression is provided. The method includes receiving an original texture and an original mesh associated with an object. The method includes generating a plurality of compressed textures based on the original texture, wherein each compressed texture of the plurality of compressed textures has a respective compression rate, and training a machine learning (ML) model based on the original mesh. For each compressed texture of the plurality of compressed textures, the method includes applying the ML model to a compressed texture and the original texture to generate a mesh estimate associated with the compressed texture, determining an error parameter between the mesh estimate associated with the compressed texture and the originalmesh and, in response to the error parameter not exceeding a predefined threshold, determining an estimated network performance requirement based on a data amount of the compressed texture, a latency constraint, and a resolution or Quality of Experience (QoE) requirement. The method further includes selecting a selected compressed texture from the plurality of compressed textures that has a lowest bit rate that also has an estimated network performance level that is less than a current network performance level, and transmitting the selected compressed texture and the mesh estimate to a wireless communication device.L0006] In an embodiment, the error parameter is a difference between the mesh estimate and the original mesh according to a predefined error metric comprising at least one of Mean Squared Error or a Chamfer distance between key points.

[0007] In an embodiment, in response to the estimated network performance level being greater than the current network performance level, the method further comprises updating one or more predefined settings. The updating can include updating an optimal network performance score with the current network performance level, updating a lowest compression rate, or updating a best mesh with a current mesh estimate.

[0008] In an embodiment, the plurality of compressed textures are generated at predefined compression rates.

[0009] In an embodiment, a number of compression rates of the predefined compression rates decreases after an initial selection of the selected compressed texture.

[0010] In an embodiment, the number of compression rates of the predefined compression rates resets after a predefined length of time.

[0011] In an embodiment, the number of compression rates of the predefined compression rates increases in response to rotation or translation of the object exceeding a predefined threshold.

[0012] In an embodiment, the number of compression rates of the predefined compression rates increases in response to a luminosity of the original texture changing more than a predefined threshold.

[0013] In an embodiment, the number of compression rates of the predefined compression rates increases in response to a difference in bit rate of different compressed textures exceeding a predefined threshold.

[0014] In an embodiment, a server is provided that is configured to perform network- optimized texture-assisted mesh compression. The server comprises processing circuitry configured to cause the server to perform any of the embodiments described above.

[0015] In an embodiment, a computer program is provided that comprises instructions which, when executed on at least one processor, cause the processor to carry out the method according to any of the embodiments above. A carrier containing the computer program is also provided, where the carrier is one of an electronic signal, an optical signal, a radio signal, or a memory.

[0016] In an embodiment, some of the advantages of the techniques proposed herein include optimization along two important dimensions:• Data complexity: the method proposed herein utilizes off-the-shelf learning -based texture-to-mesh algorithms to generate candidate compressed meshes, thus bypassing traditionally heavier mesh compression algorithms to align with compressed textures.• Network optimization: the method proposed herein enables the dynamic adjustment of the compression rate based on available network resources while maintaining, if possible, a minimum viable quality of experience (QoE). This method takes radio resources on the network into account when selecting candidate texture -mesh pairs and compression rates, giving applications more resilience to network bottlenecks.BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawing figures incorporated in and forming a part of this specification illustrate several aspects of the disclosure, and together with the description serve to explain the principles of the disclosure.

[0018] Figure 1 is an exemplary block diagram of a system for performing network- optimized texture-assisted mesh compression according to some embodiments of the present disclosure;

[0019] Figure 2 is a flowchart of a method for performing network-optimized texture- assisted mesh compression according to some embodiments of the present disclosure;

[0020] Figure 3 illustrates one example of a cellular communications system according to some embodiments of the present disclosure;

[0021] Figure 4 illustrates a wireless communication system represented as a 5G network architecture composed of core Network Functions according to some embodiments of the present disclosure;

[0022] Figure 5 is a schematic block diagram of a server according to some embodiments of the present disclosure;

[0023] Figure 6 is a schematic block diagram that illustrates a virtualized embodiment of the server of Figure 5 according to some embodiments of the present disclosure; and

[0024] Figure 7 is a schematic block diagram of the server of Figure 5 according to some other embodiments of the present disclosure.DETAILED DESCRIPTION

[0025] The embodiments set forth below represent information to enable those skilled in the art to practice the embodiments and illustrate the best mode of practicing the embodiments. Upon reading the following description in light of the accompanying drawing figures, those skilled in the art will understand the concepts of the disclosure and will recognize applications of these concepts not particularly addressed herein. It should be understood that these concepts and applications fall within the scope of the disclosure.

[0026] Server: As used herein, a server is a computational device or devices capable of running predictive mesh algorithms across candidate textures. The framework currently assumes that this process is offloaded from a computationally constrained client device. The server could be implemented in part or in whole by a radio access network node or a core network node of a Third Generation Partnership Program (3GPP) cellular communications system. The server could also be implemented in the cloud, or across distributed devices communicably coupled to a client device.

[0027] Wireless Communication Device: One type of communication device is a wireless communication device, which may be any type of wireless device that has access to (i.e., is served by) a wireless network (e.g., a cellular network). Some examples of a wireless communication device include, but are not limited to: a User Equipment device (UE) in a 3GPP network, a Machine Type Communication (MTC) device, and an Internet of Things (loT) device. Such wireless communication devices may be, or may be integrated into, a mobile phone, smart phone, sensor device, meter, vehicle, household appliance, medical appliance, media player, camera, or any type of consumer electronic, for instance, but not limited to, a television, radio, lighting arrangement, tablet computer, laptop, or PC. The wireless communication device may be a portable, hand-held, computer-comprised, or vehicle-mounted mobile device, enabled to communicate voice and / or data via a wireless connection. In an embodiment, the wireless communication device receives the compressed texture and generated mesh at the optimized compression rate. This device may have computational constraints that require this process to be optimized at the server level.

[0028] Note that the description given herein focuses on a 3GPP cellular communications system and, as such, 3GPP terminology or terminology similar to 3GPP terminology is oftentimes used. However, the concepts disclosed herein are not limited to a 3GPP system.

[0029] Note that, in the description herein, reference may be made to the term “cell”; however, particularly with respect to 5G NR concepts, beams may be used instead of cells and, as such, it is important to note that the concepts described herein are equally applicable to both cells and beams.

[0030] Some of the challenges facing the current transmission of 3D modeling technology is that point cloud compression today does not take into account hints and information available in the texture, but is done independent from this. Therefore, a larger data transmission is required when transmitting the three dimensional (3D) mesh and texture to another device than would be required if the point cloud compression used these hints to optimize the streamed immersive media experiences.

[0031] The techniques proposed herein solve these challenges by utilizing the fact that shapes and texture have common information and that the texture in a 3D model contains information that can be used to predict geometry. By performing mesh compression in conjunction with a texture compression, it is possible to achieve compressed texture and mesh pairs without undergoing costly mesh compression and alignment processes. Off-the-shelf methods can be used in innovative ways for generating compressed meshes from textures to generate series of candidate compression rates and corresponding mesh models based on current or predicted network and UE conditions. Based on candidates that fit the minimum viable quality criteria, a selection algorithm can be introduced for determining the best candidate mesh to be transmitted.

[0032] In an embodiment, the generation of candidate texture / mesh pairs corresponding to fixed or dynamic compression rates corresponding to predicted or actual network conditions for evaluation in this algorithm. The estimation of a network performance score for compressed texture and mesh candidates at ratios corresponding to predicted (or actual) network conditions. The iterative refinement of this network performance score can be based on use case specifications for quality of mesh and texture, use case or media specifications for the precision required for mesh and texture alignment, minimum resolution requirements, or minimum performance requirements for given use cases.

[0033] Various embodiments described herein provide for a method to perform network- optimized texture-assisted mesh compression. The method includes receiving an original texture and an original mesh associated with an object. The method includes generating a plurality of compressed textures based on the original texture, wherein each compressed texture of the plurality of compressed textures has a respective compression rate and training a Machine Learning (ML) model based on the original mesh. For each compressed texture of the pluralityof compressed textures, the method includes applying the ML model to a compressed texture and the original texture to generate a mesh estimate associated with the compressed texture, determining an error parameter between the mesh estimate associated with the compressed texture and the original mesh, and in response to the error parameter not exceeding a predefined threshold, determining an estimated network performance requirement based on a data amount of the compressed texture, a latency constraint, and a resolution or Quality of Experience (QoE) requirement. The method further includes selecting a selected compressed texture from the plurality of compressed textures that has a lowest bit rate that also has an estimated network performance level that is less than a current network performance level, and transmitting the selected compressed texture and the mesh estimate to a wireless communication device.

[0034] Some of the advantages of the techniques proposed herein include optimization along two important dimensions:• Data complexity: the method proposed herein utilizes off-the-shelf learning-based texture-to-mesh algorithms to generate candidate compressed meshes, thus bypassing traditionally heavier mesh compression algorithms to align with compressed textures.• Network optimization: the method proposed herein enables the dynamic adjustment of the compression rate based on available network resources while maintaining, if possible, a minimum viable quality of experience (QoE). This method takes radio resources on the network into account when selecting candidate texture-mesh pairs and compression rates, giving applications more resilience to network bottlenecks.

[0035] Figure 1 is an exemplary block diagram of a system for performing network- optimized texture-assisted mesh compression according to some embodiments of the present disclosure.

[0036] The block diagram in Figure 1 depicts some of the inputs and outputs of the method in visual form, and is presented for ease of understanding, but the organization of these elements is merely exemplary.

[0037] A three dimensional (3D) mesh 102 and a texture 104 can be received by the server 500. The mesh 102 can be the structural build of a three-dimensional model consisting of polygons. 3D meshes use reference points in X, Y and Z axes to define shapes with height, width and depth. Texture 104 can be the small-scale structure of an image that's based on the spatial arrangement of color or intensities.

[0038] The server 500 can generate a set of compressed textures 106-1, 106-2, and 106-3 (can be collectively referred to as compressed textures 106). Each of the compressed textures 106 can be created with different compression rates, such that each of the compressed textures106 has a different size or bit rate associated with the compressed textures 106. The compressed textures 106 can be generated using a pre-specified compression algorithm or series of compression algorithms at different compression rates. The different compression rates for the compressed textures 106-1, 106-2, and 106-3 can be predefined and can be based on network performance, required resolutions or Quality of Experience (QoE), or latency concerns. The different compression rates can be dynamically adjusted based on changing variables or based on predefined time periods.L0039] The original mesh 102 can be pre-processed by a ML model 108 108 for learningbased mesh prediction. In various examples, off the shelf software packages can be used. The ML model 108 could have a rotational Convolutional Neural Network (CNN) architecture, or could be any other neural network model such as a visual transformer model or other models.

[0040] For each compressed texture (e.g., each of compressed textures 106-1, 106-2, and 106-3) the server 500 can load the compressed texture, preprocess the compressed texture using the ML model 108, and then estimate a mesh estimate from the original texture 104 and the compressed textures 106. The output mesh estimates 110-1, 110-2, and 110-3 can correspond to the respective compressed textures 106-1, 106-2, and 106-3.

[0041] The mesh estimates 1 10-1, 110-2, and 110-3 are all meshes that are generated by the ML model 108 that are based on the compressed textures 106-1, 106-2, and 106-3, and each have some inherent error relative to the compressed textures 106-1, 106-2, and 106-3 after the ML model 108 applies the mesh prediction that was trained using the original mesh 102. The error filter 112 can determine the amount of error by first defining a suitable error metric (e.g., Mean Square Error (MSE) or Chamfer distance between key points) and then compute an error parameter for each of the mesh estimates 110-1, 110-2, and 110-3 based on the error metric. There can be a predefined error threshold, where if the mesh estimate error parameter exceeds the error threshold, the mesh estimate is discarded. In Figure 1 as an example, the mesh estimate 110-1 associated with compressed texture 106-1 exceeds the error metric and is thus discarded, while mesh estimates 110-2 and 110-3 are kept.

[0042] The server 500 can also estimate network performance based on the data size of compressed artifacts, available bandwidth, latency constraints, and the required resolution or QoE requirements at the client 116 end. To determine the estimated network performance, the server 500 can determine the required bandwidth and latency for transmitting the current compressed texture. The server 500 can then compare the required bandwidth and latency with the available bandwidth and latency constraints and then calculate a network performance score (e.g., a weighted sum of the error, bandwidth, and latency based on use case constraints).

[0043] The rate comparison component 115 can then select a compressed texture from amongst compressed textures 106-2 or 106-3 from the plurality of compressed textures (106) that has a lowest bit rate that also has an estimated network performance level that is less than a current network performance level.

[0044] Once the compressed texture (e.g., 106-2) is selected, the server 500 can transmit the compressed texture 106-2 along with the associated mesh estimate 110-2 to the client 116.

[0045] In an embodiment, if the network performance score is better than the current network performance, the server 500 can update some of the predefined settings, including the optimal network performance score with the current network performance level, a lowest compression rate, or a best mesh with a current mesh estimate.

[0046] In some embodiments, the list of proposed compression rates does not necessarily have to be static but could be created dynamically after the initial level has been chosen. If the model and lighting conditions can be assumed to be static, the number of candidates can shrink over time and with less and less deviations in the bitrate / compression settings. Every Nth loop, this list can grow again (temporarily) to confirm that the initial assumption is still valid. The amount of fluctuation in number of prediction levels can be controlled by the amount of movement of the model / item itself or maybe some internal diff-metric based on some of the compression comparisons.

[0047] In one embodiment for example, if estimated “cardinal directions” of model has moved more than 45 degrees in any direction, this would trigger an increase in number of prediction levels. In another embodiment, if average luminosity of the texture changes with more than X (e.g., 20%) the server 500 can increase the number of prediction levels. In another embodiment, if the difference in bitrate between any of the compressed textures or mesh estimates is larger than 100% of the higher one, the server 500 can increase the number of levels.

[0048] In an embodiment, reducing the number of possible compression levels reduce the time it takes to compute the total loop and the energy necessary to produce the compression, and thus the number of compression levels can be reduced if the object or network performance is stable, but if there are variances, the compression levels can increase for a time period to reflect the new context.

[0049] Figure 2 is a flowchart of a method for performing network-optimized texture- assisted mesh compression according to some embodiments of the present disclosure.

[0050] The method of Figure 2 can begin at step 202 where the method includes receiving an original texture 104 and an original mesh 102 associated with an object. The server 500 canthen, at step 204, generate a plurality of compressed textures 106 based on the original texture, wherein each compressed texture 106-1, 106-2, 106-3 of the plurality of compressed textures has a respective compression rate. The plurality of compressed textures 106 can be generated at predefined compression rates. In an embodiment, a number of compression rates of the predefined compression rates decreases after an initial selection of the selected compressed texture. Additionally, in an embodiment, the number of compression rates of the predefined compression rates resets after a predefined length of time, or the number of compression rates of the predefined compression rates increases in response to rotation or translation of the object exceeding a predefined threshold. In an embodiment, the number of compression rates of the predefined compression rates increases in response to a luminosity of the original texture 104 changing more than a predefined threshold. In an embodiment, the number of compression rates of the predefined compression rates increases in response to a difference in bit rate of different compressed textures 106 exceeding a predefined threshold.

[0051] At step 206, the method includes training a ML model 108 based on the original mesh 102.

[0052] And then, for each compressed texture of the plurality of compressed textures, the server 500 can at step 208 apply the ML model 108 to a compressed texture 106-2 and the original texture 104 to generate a mesh estimate (e.g., 110-2, 110-3, and 110-1) associated with the compressed texture. At step 210, the server can then, for each compressed texture, determine an error parameter between the mesh estimate (e.g., 110-1, 110-2, and 110-3) and the original mesh 102. In response to the error parameter not exceeding a predefined threshold, at step 212 the method includes determining 212 an estimated network performance requirement based on a data amount of the compressed texture, a latency constraint, and a resolution or QoE requirement. In an embodiment, the error parameter is a difference between the mesh estimate 110-2 and the original mesh 102 according to a predefined error metric comprising at least one of Mean Squared Error or a Chamfer distance between key points.

[0053] Then, from the mesh estimates that pass the filter of the error parameter not exceeding the predefined threshold (e.g., mesh estimate 110-2 and 110-3), the server 500 at step 214 selects a selected compressed texture (e.g., 106-2) from the plurality of compressed textures 106 that has a lowest bit rate that also has an estimated network performance level that is less than a current network performance level. Then, at step 216, the server transmits, or causes to be transmitted, the selected compressed texture 106-2 and the mesh estimate 110-2 to a client 116 (e.g., a wireless communication device).

[0054] At step 218, in response to the estimated network performance level being greaterthan the current network performance level, the method further includes updating one or more predefined settings, including updating 220 an optimal network performance score with the current network performance level, updating 222 a lowest compression rate, or updating 224 a best mesh with a current mesh estimate.

[0055] Figure 3 illustrates one example of a cellular communications system 300 in which embodiments of the present disclosure may be implemented. In the embodiments described herein, the cellular communications system 300 is a 5G system (5GS) including a Next Generation RAN (NG-RAN) and a 5G Core (5GC) or an Evolved Packet System (EPS) including an Evolved Universal Terrestrial RAN (E-UTRAN) and an Evolved Packet Core (EPC). In this example, the RAN includes base stations 302-1 and 302-2, which in the 5GS include NR base stations (gNBs) and optionally next generation eNBs (ng-eNBs) (e.g., LTE RAN nodes connected to the 5GC) and in the EPS include eNBs controlling corresponding (macro) cells 304-1 and 304-2. The base stations 302-1 and 302-2 are generally referred to herein collectively as base stations 302 and individually as base station 302. Likewise, the (macro) cells 304-1 and 304-2 are generally referred to herein collectively as (macro) cells 304 and individually as (macro) cell 304. The RAN may also include a number of low power nodes 306-1 through 306-4 controlling corresponding small cells 308-1 through 308-4. The low power nodes 306- 1 through 306-4 can be small base stations (such as pico or femto base stations) or RRHs, or the like. Notably, while not illustrated, one or more of the small cells 308-1 through 308-4 may alternatively be provided by the base stations 302. The low power nodes 306-1 through 306-4 are generally referred to herein collectively as low power nodes 306 and individually as low power node 306. Likewise, the small cells 308-1 through 308-4 are generally referred to herein collectively as small cells 308 and individually as small cell 308. The cellular communications system 300 also includes a core network 310, which in the 5G System (5GS) is referred to as the 5GC. The base stations 302 (and optionally the low power nodes 306) are connected to the core network 310.

[0056] The server 500 that implements the systems and methods depicted in Figure 1 and 2 can be implemented, for example, by the base stations 302 or the low power nodes 306 or by the core network 310. The wireless communications devices 312 to which the base stations 302 and low power nodes 306 are communicably coupled can be the client device 1 16 to which the compressed texture and mesh are sent.

[0057] The base stations 302 and the low power nodes 306 provide service to wireless communication devices 312-1 through 312-5 in the corresponding cells 304 and 308. The wireless communication devices 312-1 through 312-5 are generally referred to hereincollectively as wireless communication devices 312 and individually as wireless communication device 312. In the following description, the wireless communication devices 312 are oftentimes UEs, but the present disclosure is not limited thereto.

[0058] Figure 4 illustrates a wireless communication system represented as a 5G network architecture composed of core Network Functions (NFs), where interaction between any two NFs is represented by a point-to-point reference point / interface. Figure 4 can be viewed as one particular implementation of the system 300 of Figure 3.

[0059] Seen from the access side, the 5G network architecture shown in Figure 4 comprises a plurality of UEs 312 connected to either a RAN 302 or an Access Network (AN) as well as an AMF 400. Typically, the R(AN) 302 comprises base stations, e.g. such as eNBs or gNBs or similar. Seen from the core network side, the 5GC NFs shown in Figure 4 include a NSSF 402, an AUSF 404, a UDM 406, the AMF 400, a SMF 408, a PCF 410, and an Application Function (AF) 412. In various embodiments, the server 500 can be implemented by one or more of the core network nodes depicted in Figure 4.

[0060] Reference point representations of the 5G network architecture are used to develop detailed call flows in the normative standardization. The N 1 reference point is defined to carry signaling between the UE 312 and AMF 400. The reference points for connecting between the AN 302 and AMF 400 and between the AN 302 and UPF 414 are defined as N2 and N3, respectively. There is a reference point, Ni l, between the AMF 400 and SMF 408, which implies that the SMF 408 is at least partly controlled by the AMF 400. N4 is used by the SMF 408 and UPF 414 so that the UPF 414 can be set using the control signal generated by the SMF 408, and the UPF 414 can report its state to the SMF 408. N9 is the reference point for the connection between different UPFs 414, and N14 is the reference point connecting between different AMFs 400, respectively. N15 and N7 are defined since the PCF 410 applies policy to the AMF 400 and SMF 408, respectively. N12 is required for the AMF 400 to perform authentication of the UE 312. N8 and N10 are defined because the subscription data of the UE 312 is required for the AMF 400 and SMF 408.

[0061] The 5GC network aims at separating UP and CP. The UP carries user traffic while the CP carries signaling in the network. In Figure 4, the UPF 414 is in the UP and all other NFs, i.e., the AMF 400, SMF 408, PCF 410, AF 412, NSSF 402, AUSF 404, and UDM 406, are in the CP. Separating the UP and CP guarantees each plane resource to be scaled independently. It also allows UPFs to be deployed separately from CP functions in a distributed fashion. In this architecture, UPFs may be deployed very close to UEs to shorten the Round Trip Time (RTT) between UEs and data network for some applications requiring low latency.

[0062] The core 5G network architecture is composed of modularized functions. For example, the AMF 400 and SMF 408 are independent functions in the CP. Separated AMF 400 and SMF 408 allow independent evolution and scaling. Other CP functions like the PCF 410 and AUSF 404 can be separated as shown in Figure 4. Modularized function design enables the 5GC network to support various services flexibly.

[0063] Each NF interacts with another NF directly. It is possible to use intermediate functions to route messages from one NF to another NF. In the CP, a set of interactions between two NFs is defined as service so that its reuse is possible. This service enables support for modularity. The UP supports interactions such as forwarding operations between different UPFs.

[0064] Figure 5 is a schematic block diagram of a server 500 that implements the method described in Figure 1 and Figure 2 according to some embodiments of the present disclosure. Optional features are represented by dashed boxes. The server 500 may be, for example, a base station 302 or 306 or a network node that implements all or part of the functionality of the base station 302 or gNB described herein. The server node 500 may also be one or more of the core network nodes, such as AF 412. The server node 500 may also be a cloud based server, not associated with a 3GPP system. As illustrated, the server 500 includes a control system 502 that includes one or more processors 504 (e.g., Central Processing Units (CPUs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), and / or the like), memory 506, and a network interface 508. The one or more processors 504 are also referred to herein as processing circuitry. The one or more processors 504 operate to provide one or more functions of a server 500 as described herein. In some embodiments, the function(s) are implemented in software that is stored, e.g., in the memory 506 and executed by the one or more processors 504.

[0065] Figure 6 is a schematic block diagram that illustrates a virtualized embodiment of the server 500 according to some embodiments of the present disclosure. This discussion is equally applicable to other types of network nodes. Further, other types of network nodes may have similar virtualized architectures. Again, optional features are represented by dashed boxes.

[0066] As used herein, a “virtualized” server is an implementation of the server 500 in which at least a portion of the functionality of the server 500 is implemented as a virtual component(s) (e.g., via a virtual machine(s) executing on a physical processing node(s) in a network(s)). As illustrated, in this example, the server 500 may include the control system 502 as described above. The server 500 includes one or more processing nodes 600 coupled to or included as part of a network(s) 602. If present, the control system 502 are connected to theprocessing node(s) 600 via the network 602. Each processing node 600 includes one or more processors 604 (e.g., CPUs, ASICs, FPGAs, and / or the like), memory 606, and a network interface 608.

[0067] In this example, functions 610 of the server 500 described herein are implemented at the one or more processing nodes 600 or distributed across the one or more processing nodes 600 and the control system 502 in any desired manner. In some particular embodiments, some or all of the functions 610 of the server 500 described herein are implemented as virtual components executed by one or more virtual machines implemented in a virtual environment(s) hosted by the processing node(s) 600. As will be appreciated by one of ordinary skill in the art, additional signaling or communication between the processing node(s) 600 and the control system 502 is used in order to carry out at least some of the desired functions 610.

[0068] In some embodiments, a computer program including instructions which, when executed by at least one processor, causes the at least one processor to carry out the functionality of server 500 or a node (e.g., a processing node 600) implementing one or more of the functions 610 of the server 500 in a virtual environment according to any of the embodiments described herein is provided. In some embodiments, a carrier comprising the aforementioned computer program product is provided. The carrier is one of an electronic signal, an optical signal, a radio signal, or a computer readable storage medium (e.g., a non-transitory computer readable medium such as memory).

[0069] Figure 7 is a schematic block diagram of the server 500 according to some other embodiments of the present disclosure. The server 500 includes one or more modules 700, each of which is implemented in software. The module(s) 700 provide the functionality of the server 500 described herein. This discussion is equally applicable to the processing node 600 of Figure 6 where the modules 700 may be implemented at one of the processing nodes 600 or distributed across multiple processing nodes 600 and / or distributed across the processing node(s) 600 and the control system 502.

[0070] Any appropriate steps, methods, features, functions, or benefits disclosed herein may be performed through one or more functional units or modules of one or more virtual apparatuses. Each virtual apparatus may comprise a number of these functional units. These functional units may be implemented via processing circuitry, which may include one or more microprocessor or microcontrollers, as well as other digital hardware, which may include Digital Signal Processors (DSPs), special-purpose digital logic, and the like. The processing circuitry may be configured to execute program code stored in memory, which may include one or several types of memory such as Read Only Memory (ROM), Random Access Memory (RAM),cache memory, flash memory devices, optical storage devices, etc. Program code stored in memory includes program instructions for executing one or more telecommunications and / or data communications protocols as well as instructions for carrying out one or more of the techniques described herein. In some implementations, the processing circuitry may be used to cause the respective functional unit to perform corresponding functions according to one or more embodiments of the present disclosure.

[0071] While processes in the figures may show a particular order of operations performed by certain embodiments of the present disclosure, it should be understood that such order is exemplary (e.g., alternative embodiments may perform the operations in a different order, combine certain operations, overlap certain operations, etc.).

[0072] Those skilled in the art will recognize improvements and modifications to the embodiments of the present disclosure. All such improvements and modifications are considered within the scope of the concepts disclosed herein.

Claims

CLAIMS1. A method implemented in a server (500) to perform network-optimized texture assisted mesh compression, the method comprising: receiving (202) an original texture (104) and an original mesh (102) associated with an object; generating (204) a plurality of compressed textures (106) based on the original texture, wherein each compressed texture (106-1, 106-2, 106-3) of the plurality of compressed textures (106) has a respective compression rate; training (206) a machine learning, ML, model (108) based on the original mesh (102); for each compressed texture (106-1, 106-2, 106-3) of the plurality of compressed textures (106): applying (208) the ML model (108) to a compressed texture (106-2) and the original texture (104) to generate a mesh estimate (110-2) associated with the compressed texture (106-2); determining (210) an error parameter between the mesh estimate (110-2) associated with the compressed texture (106-2) and the original mesh (102); and in response to the error parameter not exceeding a predefined threshold, determining (212) an estimated network performance requirement based on a data amount of the compressed texture, a latency constraint, and a resolution or Quality of Experience, QoE, requirement; selecting (214) a selected compressed texture (106-2) from the plurality of compressed textures (106) that has a lowest bit rate that also has an estimated network performance level that is less than a current network performance level; and transmitting (216) the selected compressed texture (106-2) and the mesh estimate (110-2) to a wireless communication device.

2. The method of claim 1 , wherein the error parameter is a difference between the mesh estimate (110-2) and the original mesh (102) according to a predefined error metric comprising at least one of Mean Squared Error or a Chamfer distance between key points.

3. The method of any of claims 1 to 2, wherein in response to the estimated network performance level being greater than the current network performance level, the method further comprises:updating (218) one or more predefined settings.

4. The method of claim 3, wherein the updating the predefined settings comprises: updating (220) an optimal network performance score with the current network performance level; updating (222) a lowest compression rate; and updating (224) a best mesh with a current mesh estimate (110-2).

5. The method of any of claims 1 to 4, wherein the plurality of compressed textures (106) are generated at predefined compression rates.

6. The method of claim 5, wherein a number of compression rates of the predefined compression rates decreases after an initial selection of the selected compressed texture.

7. The method of claim 6, wherein the number of compression rates of the predefined compression rates resets after a predefined length of time.

8. The method of claim 6, wherein the number of compression rates of the predefined compression rates increases in response to rotation or translation of the object exceeding a predefined threshold.

9. The method of claim 6, wherein the number of compression rates of the predefined compression rates increases in response to a luminosity of the original texture (104) changing more than a predefined threshold.

10. The method of claim 6, wherein the number of compression rates of the predefined compression rates increases in response to a difference in bit rate of different compressed textures (106) exceeding a predefined threshold.

11. A server (500) configured to perform network-optimized texture assisted mesh compression, the server (500) comprising processing circuitry configured to cause the server (500) to: receive (202) an original texture (104) and an original mesh (102) associated with an object;generate (204) a plurality of compressed textures (106) based on the original texture, wherein each compressed texture (106-1, 106-2, 106-3) of the plurality of compressed textures (106) has a respective compression rate; train (206) a machine learning, ML, model (108) based on the original mesh (102); for each compressed texture (106-1, 106-2, 106-3) of the plurality of compressed textures (106): apply (208) the ML model (108) to a compressed texture (106-2) and the original texture (104) to generate a mesh estimate (110-2) associated with the compressed texture (106-2); determine (210) an error parameter between the mesh estimate (110-2) associated with the compressed texture (106-2) and the original mesh (102); and in response to the error parameter not exceeding a predefined threshold, determine (212) an estimated network performance requirement based on a data amount of the compressed texture, a latency constraint, and a resolution or Quality of Experience, QoE, requirement; select (214) a selected compressed texture (106-2) from the plurality of compressed textures (106) that has a lowest bit rate that also has an estimated network performance level that is less than a current network performance level; and transmit (216) the selected compressed texture (106-2) and the mesh estimate (110-2) to a wireless communication device.

12. The server (500) of claim 11 , wherein the error parameter is a difference between the mesh estimate (110-2) and the original mesh (102) according to a predefined error metric comprising at least one of Mean Squared Error or a Chamfer distance between key points.

13. The server (500) of any of claims 11 to 12, wherein in response to the estimated network performance level being greater than the current network performance level, the processing circuitry further causes the server (500) to: update (218) one or more predefined settings.

14. The server (500) of claim 13, wherein the updating the predefined settings comprises: updating (220) an optimal network performance score with the current network performance level; updating (222) a lowest compression rate; andupdating (224) a best mesh with a current mesh estimate (110-2).

15. The server (500) of any of claims 11 to 14, wherein the plurality of compressed textures (106) are generated at predefined compression rates.

16. The server (500) of claim 15, wherein a number of compression rates of the predefined compression rates decreases after an initial selection of the selected compressed texture.

17. The server (500) of claim 16, wherein the number of compression rates of the predefined compression rates resets after a predefined length of time.

18. The server (500) of claim 16, wherein the number of compression rates of the predefined compression rates increases in response to rotation or translation of the object exceeding a predefined threshold.

19. The server (500) of claim 16, wherein the number of compression rates of the predefined compression rates increases in response to a luminosity of the original texture (104) changing more than a predefined threshold.

20. The server (500) of claim 16, wherein the number of compression rates of the predefined compression rates increases in response to a difference in bit rate of different compressed textures (106) exceeding a predefined threshold.

21. A computer program comprising instructions which, when executed on at least one processor, cause the processor to carry out the method according to any of claims 1 to 10.

22. A carrier containing the computer program of claim 21 , wherein the carrier is one of an electronic signal, an optical signal, a radio signal, or a memory.

Images