Deployment scenarios for content guidance in 5G media streaming

Abstract

The present invention provides a method and apparatus, comprising computer code for causing at least one processor to perform the following operations: generating at least one 5G media downlink stream instance by a trusted distribution network (DN) of a mobile network operator (MNO), the at least one 5GMSd instance being configured to deliver at least a portion of content to a user equipment (UE) including a 5GMSd sensing application and a 5GMSd client, the at least portion of the content including video data; obtaining a presentation list published by a 5GMSd application provider of an external DN outside of both the trusted DN and the UE by the trusted DN; and performing a conversion between a visual media file of the content and a bitstream of the visual media data of the content according to format rules, and based on the conversion, performing at least one of streaming the content to the UE and playing the content by the UE.

Description

Cross-references to related applications

[0001] This application claims priority to U.S. Provisional Application No. 63 / 646,683, filed May 13, 2024, and U.S. Application No. 19 / 203,445, filed May 9, 2025, the entire disclosure of which is expressly incorporated herein by reference. Technical Field

[0002] This invention defines a deployment scenario for content-driven media streaming applications and services in a 5G network. Background Technology

[0003] 3GPP TS26.501 defines the overall architecture of 5G media streaming (5GMS). This specification and related specifications lack support for the content bootstrapping recently introduced in ETSI TS 103 998. The current 5GMS architecture defined in 3GPP TS 26.501 only defines the overall architecture and does not provide specific architectures for uplink and downlink media streaming.

[0004] Furthermore, for the reasons mentioned above, there is an urgent need for technical solutions to these problems in video codec technology. This document focuses on providing deployment scenarios for content-driven media streaming applications and services using 5G networks. Summary of the Invention

[0005] A video decoding method is provided, executed by at least one processor, the method comprising: generating at least one 5G Media Downlink Streaming (5GMSd) instance by a trusted distribution network (DN) of a mobile network operator (MNO), the at least one 5GMSd instance being configured to deliver at least a portion of content to a user equipment (UE), the UE including a 5GMSd-aware application and a 5GMSd client, the at least a portion of the content including video data; obtaining a presentation manifest published by a 5GMSd application provider (AP) of an external DN outside of both the trusted DN and the UE by the trusted DN; and streaming the content to the UE based on the presentation manifest and decoding the content at the UE.

[0006] A video encoding method is also provided, executed by at least one processor, the method comprising: generating at least one 5GMSd instance by a trusted DN of an MNO, the at least one 5GMSd instance being configured to deliver at least a portion of content to a UE, the UE including a 5GMSd-aware application and a 5GMSd client, the at least a portion of the content including video data; obtaining a presentation manifest published by a 5GMSd AP of an external DN outside of both the trusted DN and the UE by the trusted DN; and encoding the content and streaming the content to the UE based on the presentation manifest.

[0007] A method for processing visual media data is also provided, executed by at least one processor, the method comprising: generating at least one 5GMSd instance by a trusted DN of an MNO, the at least one 5GMSd instance being configured to deliver at least a portion of content to a UE, the UE including a 5GMSd sensing application and a 5GMSd client, the at least a portion of the content including video data; obtaining a presentation list published by a 5GMSd AP of an external DN outside of both the trusted DN and the UE by the trusted DN; and performing a conversion between a visual media file of the content and a bitstream of the visual media data of the content according to format rules, and based on the conversion, performing at least one of streaming the content to the UE and playing the content by the UE.

[0008] This presentation list can be obtained from the 5GMSd AP by the trusted DN via the M2d application programming interface (API).

[0009] Streaming the content may include: streaming at least a portion of the content from the 5GMSd AS to the UE via the M4d API.

[0010] The MNO can provide multiple 5GMSd instances, including at least one of the 5GMSd instances, which are configured to deliver the content to the UE together.

[0011] The external DN can provide at least one other 5GMSd instance, which is configured to deliver the content to the UE together with the at least one 5GMSd instance of the trusted DN.

[0012] The content streaming to the UE can be guided by the boot server of the trusted DN.

[0013] The content delivery to the UE can be guided by the boot server of the external DN. Attached Figure Description

[0014] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, wherein:

[0015] Figure 1 This is a simplified block diagram of a communication system according to an embodiment;

[0016] Figure 2 This is a simplified schematic diagram of the encoder and decoder environment according to an embodiment;

[0017] Figure 3 This is a simplified block diagram of the decoder according to an embodiment;

[0018] Figure 4 This is a simplified block diagram of the encoder according to an embodiment;

[0019] Figure 5 This is a simplified block diagram of an augmented reality (AR) system according to an embodiment;

[0020] Figure 6 This is a simplified block diagram of a 5G system according to an embodiment;

[0021] Figure 7 This is a simplified block diagram in a 5G UE environment according to an embodiment;

[0022] Figure 8 This is a simplified block diagram in an edge-dependent AR (EDGAR) environment according to an embodiment;

[0023] Figure 9 This is a simplified block diagram of the EDGAR UE environment according to an embodiment;

[0024] Figure 10 This is a simplified diagram of AR used according to the embodiment;

[0025] Figure 11 This is a simplified block diagram of a hybrid AR and non-AR system according to an embodiment;

[0026] Figure 12 This is a simplified block diagram of a non-AR UE according to an embodiment;

[0027] Figure 13 These are simplified block diagrams and timing diagrams based on embodiments;

[0028] Figure 14 This is a simplified block diagram of the 5GMS architecture according to an embodiment;

[0029] Figure 15 This is a simplified block diagram of content guidance and distribution within a trusted domain according to an embodiment;

[0030] Figure 16 This is a simplified block diagram of content guidance outside the trusted domain and hybrid distribution inside and outside the trusted domain, according to an embodiment.

[0031] Figure 17 This is a simplified block diagram guided by the content outside the DN inside the trusted domain according to an embodiment;

[0032] Figure 18 This is a simplified block diagram of content guidance within the trusted domain and some DNs outside the trusted domain, based on an embodiment.

[0033] Figure 19 This is a schematic diagram of an exemplary computer system component according to an embodiment. Detailed Implementation

[0034] The proposed features discussed below can be used individually or in any combination in any order. Furthermore, embodiments can be implemented by processing circuitry (e.g., at least one processor or at least one integrated circuit (IC)). In one example, at least one processor executes a program stored in a non-transitory computer-readable storage medium.

[0035] Figure 1 A simplified block diagram of a communication system 100 according to an embodiment of the present invention is shown. The communication system 100 may include at least two terminals 102 and 103 interconnected via a network 105. To achieve unidirectional data transmission, the first terminal 103 may encode video data locally for transmission to the other terminal 102 via the network 105. The second terminal 102 may receive the encoded video data from the other terminal from the network 105, decode the encoded data, and display the recovered video data. Unidirectional data transmission may be common in scenarios such as media service applications.

[0036] Figure 1 A second pair of terminals 101 and 104 is shown, configured to support bidirectional transmission of encoded video, which may occur, for example, during a video conference. To enable bidirectional data transmission, each terminal 101 and 104 can encode video data acquired at a local location for transmission to the other terminal via network 105. Each terminal 101 and 104 can also receive encoded video data transmitted by the other terminal, decode the encoded data, and display the recovered video data on a local display device.

[0037] exist Figure 1In this invention, terminals 101, 102, 103, and 104 may be represented as servers, personal computers, and smartphones, but the principles of the invention are not limited thereto. Embodiments of the invention can be applied to laptops, tablets, media players, and / or dedicated video conferencing equipment. Network 105 refers to any number of networks, including, for example, wired and / or wireless communication networks, that transmit encoded video data between terminals 101, 102, 103, and 104. Communication network 105 may exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet. For the purposes of this discussion, unless otherwise stated below, the architecture and topology of network 105 may not materially affect the operation of the invention.

[0038] Figure 2 As an application example of the disclosed subject matter, the deployment of a video encoder and decoder in a streaming media environment is shown. The disclosed subject matter is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0039] The streaming media system may include an acquisition subsystem 203, which may include a video source 201, such as a digital camera, which, for example, creates an uncompressed sample video stream 213. It should be emphasized that the sample stream 213 may have a higher data volume compared to the encoded video stream and may be processed by an encoder 202 coupled to the camera 201. The encoder 202 may include hardware, software, or a combination of hardware and software to enable or implement aspects of the disclosed subject matter described in more detail below. It should be emphasized that the encoded video stream 204 may have a lower data volume compared to the sample stream and may be stored on a streaming media server 205 for future use. At least one streaming media client 212 and 207 may access the streaming media server 205 to retrieve copies 208 and 206 of the encoded video stream 204. Client 212 may include video decoder 211, which decodes a copy 208 of the incoming encoded video stream and creates an output video sample stream 210, which can be rendered on display 209 or other rendering device (not depicted). In some streaming media systems, video streams 204, 206, and 208 may be encoded according to specific video encoding / compression standards. Examples of these standards have been mentioned above and will be further described in this application.

[0040] Figure 3 This can be a functional block diagram of the video decoder 300 according to an embodiment of the present invention.

[0041] Receiver 302 can receive at least one encoded video sequence to be decoded by decoder 300; in the same embodiment or another embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of other encoded video sequences. The encoded video sequences can be received from channel 301, which can be a hardware / software link to a storage device storing the encoded video data. Receiver 302 can receive the encoded video data, as well as other data that can be forwarded to the corresponding user entity (not depicted), such as encoded audio data and / or auxiliary data streams. Receiver 302 can separate the encoded video sequences from other data. To combat network jitter, buffer memory 303 can be coupled between receiver 302 and entropy decoder / resolver 304 (hereinafter referred to as the "resolver"). Buffer 303 may not be necessary or may be small when receiver 302 receives data from a store / forward device with sufficient bandwidth and controllability or from an isosynchronous network. For use on best-effort packet networks such as the Internet, buffer 303 may be necessary, can be relatively large, and can advantageously have an adaptive size.

[0042] The video decoder 300 may include a parser 304 for reconstructing symbols 313 from an entropy-coded video sequence. These symbols may include information for managing the operation of the decoder 300, and information that may be used to control rendering devices, such as a display 312, which is not part of the decoder but may be coupled to it. Control information for one or more rendering devices may be in the form of Supplementary Enhancement Information (SEI) (SEI messages) or fragments of video availability information parameter sets (not depicted). The parser 304 may perform parsing / entropy decoding on the received encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow principles well known to those skilled in the art, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser 304 may extract a subgroup parameter set from the encoded video sequence for at least one pixel subgroup in the video decoder based on at least one parameter corresponding to a group. Subgroups can include Group of Pictures (GOP), pictures, tiles, slices, macroblocks, coding units (CU), blocks, transform units (TU), prediction units (PU), etc. The entropy decoder / parser can also extract information such as transform coefficients, quantizer parameter values, and motion vectors from the encoded video sequence.

[0043] The parser 304 can perform entropy decoding / parsing operations on the video sequence received from the buffer 303 to create symbols 313. The parser 304 can receive encoded data and selectively decode specific symbols 313. Furthermore, the parser 304 can determine whether to provide specific symbols 313 to the motion compensation prediction unit 306, the scaler / inverse transform unit 305, the intra-frame prediction unit 307, or the loop filter 311.

[0044] Depending on the type of the encoded video image or its portions (e.g., inter-frame and intra-frame images, inter-frame and intra-frame blocks) and other factors, the reconstruction of symbol 313 may involve multiple different units. Which units are involved and how these units are involved can be controlled by subgroup control information parsed from the encoded video sequence by parser 304. For clarity, the flow of such subgroup control information between parser 304 and the multiple units described below is not depicted.

[0045] In addition to the functional blocks already mentioned, the decoder 300 can be conceptually subdivided into several functional units as described below. In practical implementations under commercial constraints, many of these units interact closely with each other and can be at least partially integrated with one another. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the functional units described below.

[0046] The first unit is the scaler / inverse transform unit 305. The scaler / inverse transform unit 305 receives from the parser 304 quantization transform coefficients as one or more symbols 313, along with control information including the transform to be used, block size, quantization factor, quantization scaling matrix, etc. This scaler / inverse transform unit 305 can output blocks containing sample values, which can be input to the aggregator 310.

[0047] In some cases, the output samples of the scaler / inverse transform 305 may involve intra-coded blocks; that is, the block does not use predictive information from previously reconstructed images, but instead uses predictive information from previously reconstructed portions of the current image. Such predictive information can be provided by the intra-image prediction unit 307. In some cases, the intra-image prediction unit 307 uses surrounding reconstructed information obtained from the current (partially reconstructed) image 309 to generate blocks with the same size and shape as the blocks in the reconstruction. In some cases, the aggregator 310 adds the predictive information already generated by the intra-prediction unit 307 to the output sample information provided by the scaler / inverse transform unit 305 on a per-sample basis.

[0048] In other cases, the output samples of the scaler / inverse transform unit 305 may involve inter-frame coded and possibly motion-compensated blocks. In this case, the motion-compensated prediction unit 306 can access the reference image memory 308 to obtain samples for prediction. After motion compensation of the obtained samples according to the block-related symbols 313, the aggregator 310 can add these samples to the output of the scaler / inverse transform unit (referred to in this case as residual samples or residual signals) to generate output sample information. The address at which the motion compensation unit obtains the predicted samples in the form of the reference image memory can be controlled by motion vectors, which are provided to the motion compensation unit in the form of symbols 313, which may have, for example, X, Y, and reference image components. Motion compensation may also include interpolation of sample values ​​obtained from the reference image memory when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0049] The output samples of aggregator 310 can undergo various loop filtering techniques in loop filter unit 311. Video compression techniques may include loop filtering techniques controlled by parameters included in the encoded video bitstream and used by loop filter unit 311 as symbols 313 from parser 304. However, loop filtering techniques may also respond to metadata obtained during decoding of previous (in decoding order) portions of the encoded image or encoded video sequence, as well as to previously reconstructed and loop-filtered sample values.

[0050] The output of the loop filter unit 311 can be a sample stream, which can be output to the display 312 (which can be a rendering device) and can also be stored in the reference image memory 557 for use in future inter-frame image prediction.

[0051] Some encoded images can be used as reference images for future predictions after full reconstruction. Once the encoded images are fully reconstructed and have been identified as reference images (e.g., by the parser 304), the current reference image 309 can become part of the reference image buffer 308, and the new current image memory can be reallocated before the reconstruction of subsequent encoded images begins.

[0052] The video decoder 300 can perform decoding operations according to a predetermined video compression technique, which may be documented in standards such as ITU-T H.265 Recommendation. If the encoded video sequence follows the syntax of a video compression technique or standard, such as the syntax specified in a video compression technique document or standard, particularly in its configuration file, then the encoded video sequence may conform to the syntax specified by the used video compression technique or standard. For compliance, the complexity of the encoded video sequence must also be within the limits defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum image size, maximum frame rate, maximum reconstruction sampling rate (measured, for example, in megasamples per second), maximum reference image size, etc. In some cases, the limitations set by the hierarchy can be further limited by the Hypothetical Reference Decoder (HRD) specification and the HRD buffer management metadata indicated by signaling in the encoded video sequence.

[0053] In one embodiment, receiver 302 may receive supplemental (redundant) data along with encoded video. This supplemental data may be part of one or more encoded video sequences. The supplemental data may be used by video decoder 300 to properly decode the data and / or more accurately reconstruct the original video data. The supplemental data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0054] Figure 4 This can be a functional block diagram of a video encoder 400 according to an embodiment of the present invention.

[0055] The encoder 400 can receive video samples from a video source 401 (which is not part of the encoder), which can capture one or more video images to be encoded by the encoder 400.

[0056] Video source 401 can provide a source video sequence to be encoded by encoder 303 in the form of a digital video sample stream. This digital video sample stream can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601 YCrCb, RGB, etc.), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, video source 401 can be a storage device storing previously prepared video. In a video conferencing system, video source 401 can be a camera capturing local image information as a video sequence. Video data can be provided in the form of multiple individual images, which produce an animation effect when viewed sequentially. The images themselves can be organized as spatial pixel arrays, where each pixel can include at least one sample depending on the sampling structure, color space, etc., used. Those skilled in the art will readily understand the relationship between pixels and samples. Samples are described in detail below.

[0057] According to one embodiment, encoder 400 can encode and compress images of a source video sequence into an encoded video sequence 410 in real time or under any other time constraints required by the application. Enforcing an appropriate encoding rate is one of the functions of controller 402. The controller controls and is functionally coupled to other functional units as described below. For clarity, the coupling is not depicted. Parameters set by the controller may include rate control-related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, GOP layout, maximum motion vector search range, etc. Other functions of controller 402 can be readily identified by those skilled in the art, as these functions may be related to the video encoder 400 optimized for a particular system design.

[0058] Some video encoders operate in an “encoding loop” well known to those skilled in the art. In a very simplified description, the encoding loop may consist of the encoding portion of the encoder (e.g., source encoder 403) (responsible for creating symbols based on the input image to be encoded and one or more reference images) and a (local) decoder 406 embedded in encoder 400, which reconstructs the symbols to create sample data that a (remote) decoder would also create (since in the video compression techniques considered in the disclosed subject matter, any compression between the symbols and the encoded video stream is lossless compression). The reconstructed sample stream is input to reference image memory 405. Since decoding the symbol stream produces a bit-by-bit accurate result independent of the decoder location (local or remote), the contents of the reference image buffer are also bit-by-bit accurate between the local and remote encoders. In other words, the reference image samples “seen” by the encoder’s prediction portion are exactly the same as the sample values ​​“seen” by the decoder during prediction. This fundamental principle of reference image synchronization (and the drift that occurs, for example, when synchronization cannot be maintained due to channel errors) is well known to those skilled in the art.

[0059] The operation of the "local" decoder 406 is the same as that of the "remote" decoder 300, as described above. Figure 3 The "remote" decoder 300 is described in detail. However, a brief reference is provided. Figure 4 Since symbols are available, and the entropy encoder 408 and parser 304 can encode / decode symbols into an encoded video sequence in a lossless manner, the entropy decoding portion of decoder 300, including channel 301, receiver 302, buffer 303 and parser 304, may not be fully implemented in local decoder 406.

[0060] It can be seen from this that any decoder technique other than parsing / entropy decoding, which exists in the decoder, must also exist in the corresponding encoder in essentially the same functional form. The description of encoder techniques can be simplified, as encoder techniques are the reverse operation of a fully described decoder technique. More detailed descriptions are only required in certain areas, and these descriptions are provided below.

[0061] As part of the operation of the source encoder 403, the source encoder 403 may perform motion-compensated predictive coding on the input frame, which predictively codes the input frame with reference to at least one previously encoded frame designated as a "reference frame" in the video sequence. In this way, the encoding engine 407 encodes the differences between pixel blocks of the input frame and pixel blocks of one or more reference frames, which may be selected as one or more predictive references for the input frame.

[0062] The local video decoder 406 can decode encoded video data of frames that can be designated as reference frames based on symbols generated by the source encoder 403. Advantageously, the operation of the encoding engine 407 can be a lossy process. When the encoded video data can be decoded by the video decoder ( Figure 4 During decoding (not shown), the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder 406 replicates the decoding process performed by the video decoder on the reference frame, and allows the reconstructed reference frame to be stored in a reference image memory 405, which can be, for example, a cache. In this way, the encoder 400 can store copies of the reconstructed reference frames locally, these copies having the same content (no transmission errors) as the reconstructed reference frames obtained by the remote video decoder.

[0063] Predictor 404 can perform a prediction search against encoding engine 407. That is, for a new frame to be encoded, predictor 404 can search the reference image memory 405 for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can be used as suitable prediction references for the new image. Predictor 404 can operate pixel-by-pixel based on sample blocks to find appropriate prediction references. In some cases, as determined by the search results obtained by predictor 404, the input image may have prediction references obtained from multiple reference images stored in reference image memory 405.

[0064] The controller 402 can manage the encoding operations of the video encoder 403, including, for example, setting parameters and subgroup parameters for encoding video data.

[0065] The outputs of all the aforementioned functional units can be entropy encoded in the entropy encoder 408. The entropy encoder converts the symbols generated by each functional unit into an encoded video sequence by lossless compression of the symbols according to techniques known to those skilled in the art (e.g., Huffman coding, variable-length coding, arithmetic coding, etc.).

[0066] Transmitter 409 may buffer one or more encoded video sequences created by entropy encoder 408 in preparation for transmission via communication channel 411, which may be a hardware / software link connected to a storage device where the encoded video data will be stored. Transmitter 409 may combine encoded video data from video encoder 403 with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (data sources not shown).

[0067] Controller 402 can manage the operation of encoder 400. During encoding, controller 405 can assign a specific encoded image type to each encoded image, which may affect the encoding techniques applicable to the corresponding image. For example, one of the following frame types can typically be assigned to an image:

[0068] An intra-frame image (I-image) can be an image that can be encoded and decoded without using any other frames in the sequence as a prediction source. Some video codecs allow different types of intra-frame images, including, for example, images refreshed by a standalone decoder. Those skilled in the art are aware of these variations of I-images and their corresponding applications and characteristics.

[0069] A predictive image (P-image) can be an image that is encoded and decoded by predicting sample values ​​for each block via intra-frame prediction or via inter-frame prediction using at most one motion vector and a reference index.

[0070] A bidirectional predictive image (B-image) can be an image encoded and decoded by predicting sample values ​​for each block via intra-frame prediction or via inter-frame prediction using up to two motion vectors and reference indices. Similarly, a multi-predictive image can reconstruct a single block using more than two reference images and associated metadata.

[0071] The source image can typically be spatially subdivided into multiple sample blocks (e.g., each block contains 4×4, 8×8, 4×8, or 16×16 samples) and encoded block by block. Blocks can be predictively coded with reference to other (already coded) blocks, which are determined by the coding assignments of the corresponding images applied to the block. For example, blocks of an I-image can be unpredictably coded, or these blocks can be predictively coded (spatial prediction or intra-frame prediction) with reference to already coded blocks of the same image. Pixel blocks of a P-image can be unpredictably coded, encoded via spatial prediction, or encoded via temporal prediction with reference to a previously coded reference image. Blocks of a B-image can be unpredictably coded, encoded via spatial prediction, or encoded via temporal prediction with reference to one or two previously coded reference images.

[0072] The video encoder 400 can perform encoding operations according to predetermined video coding technologies or standards such as ITU-T H.265 Recommendation. In operation, the video encoder 400 can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technology or standard used.

[0073] In one embodiment, transmitter 409 may transmit additional data along with the encoded video. Source encoder 403 may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant images and slices, SEI messages, fragments of Visual Usability Information (VUI) parameter sets, etc.

[0074] According to embodiments of this application, both the encoding and decoding processes can be considered as visual media data processing, which performs conversion between visual media files and the bitstream of visual media data according to format rules.

[0075] Figure 5 Example 500 is an end-to-end architecture for a stand-alone AR (STAR) device according to an exemplary embodiment, which illustrates a 5G STAR UE receiver 600, a network / cloud 501, and a 5G UE (transmitter) 700. Figure 6 This is a more detailed example 600 of at least one configuration for a STAR UE receiver 600 according to an exemplary embodiment, and Figure 7 This is a more detailed example 700 of at least one configuration for a 5G UE transmitter 700 according to an exemplary embodiment. 3GPP TR26.998 defines support for glasses-based augmented reality / mixed reality (AR / MR) devices in 5G networks. And according to an exemplary embodiment of this application, at least two types of devices are considered: 1) devices fully capable of decoding and playing complex AR / MR content (standalone AR or STAR), and 2) devices with limited computing resources and / or small physical size (and therefore smaller battery), which can only run such applications (edge-dependent AR or EDGAR) when most of the computation is performed on a 5G edge server, network, or cloud rather than on the device. In this application, the abbreviations Uu, Gnb, NEF, PCF, MSE, SDK, and 5GMSd can be considered as representing inter-user, gNodeB, physical control format, mobile execution environment (MExE) service environment, source deployment kit, and 5G maximum sensitivity degradation, respectively. Furthermore, the M_d in this application (such as M6d and M7d) relates to UE MSH APIs, which allow 5GMS-aware applications to interact with the 5GMSd MSH. RTP and AVP can be viewed as real-time protocol and attribute value pairs, respectively.

[0076] Furthermore, according to the exemplary embodiments described below, a shared session use case can be experienced in which all participants in the shared AR session experience have AR devices, each participant sees other participants in the AR scene, wherein the participants are presented in an overlay manner in the local physical scene, and the arrangement of the participants in the scene is consistent across all receiving devices. For example, people in each local space have the same position / seat arrangement relative to each other, and this virtual space creates the feeling of being in the same space, but since the room is the real room or space in which each person is actually located, the room varies from participant to participant.

[0077] For example, according to the combination Figures 5 to 7 In the exemplary embodiment shown, the immersive media processing function on the network / cloud 501 receives uplink streams from various devices and constructs a scene description that defines the arrangement of each participant in a single virtual meeting room. This scene description, along with the encoded media stream, is transmitted to each receiving participant. The receiving participant's 5G STAR UE 600 receives, decodes, and processes the 3D video and audio streams, and renders them using the received scene description and information received from its AR runtime, thereby creating an AR scene of the virtual meeting room that includes all other participants. While each participant's virtual room is based on their own physical space, the seating / positioning of all other participants within that room remains consistent with the virtual room of every other participant in this session.

[0078] See also, according to an exemplary embodiment, Figure 8 The example 800 illustrates an EDGAR device architecture where devices such as the 5G EDGAR UE 900 cannot perform heavy processing on their own. Therefore, scene parsing and media parsing for the received content are performed in the cloud / edge 801, and then a simplified AR scene with a small number of media components is transmitted to the device for processing and rendering. Figure 9 A more detailed example of a 5G EDGAR UE 900 according to an exemplary embodiment is shown.

[0079] Figure 10 Example 1000 is shown, in which users A10, B11, and T12 participate in an AR conference room, and at least one of these users may not have an R device. As shown, user A10 is in his office 1001, sitting in a conference room with multiple chairs, and user A10 is sitting in one of the chairs. User B11 is in his living room 1002, sitting on a small two-seater sofa, which also contains at least one two-seater sofa and other furniture such as chairs and tables. User T12 is in an airport lounge 1003, sitting on a bench opposite another bench across a coffee table, and surrounded by at least one other coffee table.

[0080] Furthermore, it can be seen that in the AR environment, within office 1001, user A 10's AR presents to user A 10 a virtual user B 11v1 corresponding to user B 11 and a virtual user T 12v1 corresponding to user T 12, and makes virtual users B 11v1 and T 12v1 appear to user A 10 as if they were sitting on furniture or an office chair in office 1001, just like user A 10. It can also be seen that in living room 1202 of example 1200, user B 11's AR presents a virtual user T 12v2 corresponding to user T 12 but sitting on a sofa in living room 1202, and a virtual user A 10v1 corresponding to user A 10 and also sitting on furniture in living room 1202 instead of an office chair in office 1201. See also airport lounge 1203, where the AR representation of user T 12 corresponds to a virtual user A10v2 of user A 10, but seated at a table in airport lounge 1203, and a virtual user B 11v2, also seated at the same table opposite virtual user A 10v2. Furthermore, in each of the offices 1201, living room 1202, and airport lounge 1203, the updated scene descriptions of each room remain consistent with the other rooms in terms of location / seating arrangement. For example, in each room, user A 10 is presented as being in a relatively counter-clockwise direction relative to user 11 or its virtual representation, while user 11 or its virtual representation is in a relatively clockwise direction relative to user T 12 or its virtual representation.

[0081] However, AR technology has always had limitations in attempting to introduce virtual space creation and use for devices that do not support AR but can parse VR or 2D video, and embodiments of this application provide an improved technical process for creating virtual scenes consistent with AR scenes when such devices participate in shared AR session services.

[0082] Figure 11 Example 1100 is shown, illustrating an end-to-end architecture with a non-AR device 1101 and a cloud / edge 1102 according to an exemplary embodiment. Figure 12 A more detailed block diagram example of a non-AR device 1101 is shown.

[0083] like Figure 11 and 12 As shown, the non-AR UE 1101 is a device capable of rendering 360-degree or 2D video but lacking any AR capabilities. However, edge functionality on the cloud / edge 1102 enables AR rendering of the received scene, the rendered scene, and immersive visual and audio objects in a virtual room selected from a library. The entire video is then encoded and transmitted to device 1101 for decoding and rendering.

[0084] Therefore, it can possess multi-view capability. For example, AR processing on the edge / cloud 1102 can generate multiple videos of the same virtual room: multiple videos with different angles and different viewports. And the device 1101 can receive at least one of these videos, switch between these videos as needed, or send commands to the edge / cloud processing to stream only the required viewport / angle.

[0085] In addition, it can also have background changing capabilities, where users on device 1101 can select the desired room background from a provided library, such as different meeting rooms or even living rooms and layouts. And the cloud / edge 1102 uses the selected background and creates a virtual room accordingly.

[0086] Figure 13 An exemplary timing diagram 1300 is shown for an exemplary call flow for receiving an immersive AR session for a non-AR UE 1101. For illustrative purposes, only one transmitter is shown in this diagram, and its detailed call flow is not shown.

[0087] The figure shows the AR application module 21, media playback module 22, and media access function module 23, which can be considered as modules receiving non-AR UE 1101. The figure also shows the cloud / edge segmentation rendering module 24. The figure further shows the media delivery module 25 and scene graph combiner module 26 of the network cloud 1102. The figure also shows the 5G transmitter UE module 700.

[0088] S1-S6 can be considered as the session establishment phase. At S1, the AR application module 21 can request to initiate a session from the media access function module 23, and at S2, the media access function module 23 can request to initiate a session from the cloud / edge segmentation rendering module 24.

[0089] The cloud / edge segmentation rendering module 24 can conduct session negotiation with the scene graph combiner module 26 at S3, and the scene graph combiner module 26 can correspondingly negotiate with the 5G transmitter UE 700. If successful, at S5, the cloud / edge segmentation rendering module can send an acknowledgment to the media access function module 23, and the media access function module 23 can send an acknowledgment to the AR application module 21.

[0090] Subsequently, S7 can be considered the media pipeline configuration phase, in which the media access function module 23 and the cloud / edge segmentation rendering module 24 each configure their respective pipelines. Then, after this pipeline configuration, a session can be initiated via the following signals: a signal sent from the AR application module to the media player module 22 at S8, a signal sent from the media player module 22 to the media access function module 23 at S9, and a signal sent from the media access function module 23 to the cloud / edge segmentation rendering module 24 at S10.

[0091] Subsequently, there may be a pose loop stage from S11 to S13, where at S11, pose data can be provided from the media player module 22 to the AR application module 21; at S12, the AR application module can provide pose data 12 to the media access function module 23, and then the media access function module 23 can provide pose data to the cloud / edge segmentation rendering module 24.

[0092] S14 to S16 can be considered as the shared experience streaming phase, wherein at S14, the 5G transmitter UE 700 can provide the media stream at S14 to the media delivery module 25, and at S15, it provides AR data to the scene graph synthesizer module 26. Subsequently, the scene graph synthesizer module 25 can synthesize at least one scene based on the received AR data, and at S16, it provides the scene and scene updates to the cloud / edge segmentation rendering module 24; simultaneously, the media delivery module 25 can provide the media stream to the cloud / edge segmentation rendering module at S17. According to an exemplary embodiment, this can include: obtaining an AR scene descriptor from a non-AR device that does not render AR scenes, and generating a virtual scene by a cloud device by parsing and rendering the scene descriptor obtained from the non-AR device.

[0093] S18 to S19 can be considered the media uplink stage, in which the media player module 22 collects and processes media data from its local user and provides the media data to the media access function module 23 at S18. Then, the media access module 23 can encode the media and provide the media stream to the cloud / edge segmentation rendering module 24 at S19.

[0094] The period between S19 and S20 can be considered the media downlink phase, in which the cloud / edge segmentation rendering module 24 performs scene parsing and completes AR rendering. S20 and S21 can then be considered the media stream loop phase. At S20, the cloud / edge segmentation rendering module 24 provides the media stream to the media access function module 23, which then decodes the media and provides the rendered media to the media player 22 at S21.

[0095] With the help of such features according to the exemplary embodiments, even if the non-AR UE 1101 lacks a perspective display and therefore cannot create an AR scene, its display can still be utilized to render VR or 2D video. Therefore, the immersive media processing function of the non-AR UE 1101 only generates a general scene description describing the relative position of each participant with respect to other participants and the scene. The scene itself needs to be adjusted at each device using pose information before it can be rendered as an AR scene as described above. Furthermore, the AR rendering process at the edge or in the cloud can parse the AR scene and create a simplified VR-2D scene.

[0096] According to an exemplary embodiment, the present invention employs a segmented rendering process similar to that of EDGAR devices for non-AR devices (such as VR or 2D video devices), characterized in that, in this case, the edge / cloud AR rendering process does not generate any AR scene. Instead, the edge / cloud AR rendering process generates a virtual scene by parsing and rendering a scene description received from an immersive media processing function for a given background (such as a conference room), and then renders each participant in the conference room at the location described by the scene description.

[0097] Furthermore, depending on the ability to receive non-AR devices, the resulting video can be a 360-degree video or a 2D video, and according to an exemplary embodiment, the generated video is generated based on the posture information received from the non-AR device.

[0098] In addition, each other participant with a non-AR device was added as a 2D video overlay to the 360-degree / 2D video of the conference room, such as... Figure 10 As shown, the meeting room can have a dedicated area for using these overlays, for example... Figure 10 Furniture areas with overlaid virtual images.

[0099] Furthermore, according to an exemplary embodiment, audio signals from all participants may be mixed as necessary to generate mono audio carrying indoor speech, video may be encoded as a single 360-degree video or 2D video and delivered to the device, and optionally, multiple video (multi-view) sources may be created, wherein each video source captures the same virtual conference room from different perspectives and provides these perspectives to the device.

[0100] In addition, the non-AR UE device 1101 can receive 360-degree video and / or at least one selected multi-view video and audio, and render them on the device display. Users can switch between different perspectives or change the viewport of the 360-degree video by moving or rotating the viewing device, thereby enabling them to roam in a virtual room while watching the video.

[0101] Although the above embodiments are equipped with such 5GMS architecture extensions to use edge servers in their architecture, and although their specifications may have many features, it has been technically impossible to deploy these features as a set of software development kits (SDKs) on the device or as a set of microservices in the cloud. This technical deficiency is addressed by the embodiments described further below.

[0102] For example, the current Media Service Enabler (MSE) technical report does not define a framework for associating specifications with SDKs, and does not include any microservices concept.

[0103] See Figure 14 Example 1400 illustrates a 5GMS architecture with edge extension according to an exemplary embodiment. As shown, a UE 1401 and a DN 1411 are present. UE 1401 may include a 5GMS client 1403 and a 5GMS-aware application 1405, such as the AR or non-AR embodiments described above, but such applications are not limited thereto. The 5GMS client 1403 may also include a media stream processor 1403 and a media session processor. DN 1411 may include a 5GMS AS 1412, a 5GMS application function (AF) 1414, and a 5GMS AP 1413. The 5GMS AF 1414 may also communicate with a network exposure function (NEF) 1415 and a policy and charging function (PCF) 1416. The improvements described in this application can be understood in the context of at least one of UE 1401, 5GMS Aware Application 1405, 5GMS AP 1413, NEF 1415, and PCF 1416; that is, instead of adopting a monolithic specification lacking an MSE definition for each function or functional group, at least one of these elements can generate its own specification and provide that specification to another element among these elements upon receiving a compliance request, which element can then further configure the initial element specification according to the various possibilities described below, and such processing can be performed through the multiple open APIs 1420 shown and at least one of interfaces M1, M2, M4, M5, M6, M7, M8, N33, and N5.

[0104] Figure 15 Example 1500 illustrates content bootstrapping and distribution within a trusted DN or domain. Figure 15 and Figure 14 Share some components, and Figure 15 Trusted DN 1501 and external DN 1502 are also shown. Trusted DN 1501 has multiple distribution servers of 5GMS AS1512, such as "Distribution Server 1" 1502, "Distribution Server 2" 1503, and boot server 1505. External DN 1502 may have, for example... Figure 14 The 5GMS AP 1413 shown.

[0105] exist Figure 15 In this collaborative scenario, the MNO provides content bootstrapping between the various DNs (internal CDNs). Content bootstrapping server 1505 also resides within trusted DN 1501.

[0106] And according to the combination Figure 15 In an exemplary embodiment, (1) the MNO may be considered to have a trusted DN 1501, in which multiple distribution servers (such as “distribution server 1” 1502 and “distribution server 2” 1503) are provided to deliver content to or from the UE 1401. (2) The MNO also provides a content bootstrapping server 1505 in the trusted DN 1501. Inventories published by application service providers (such as 5GMS AP 1413) do not have content bootstrapping information. The MNO manipulates the inventory by adding a base URL and bootstrapping server information before providing it to clients, such as at the UE 1401. (4) During streaming, the UE 1401 makes a request to the content bootstrapping server based on the information provided, and according to the exemplary embodiment, the content bootstrapping operation is performed internally by the MNO and is opaque to AP 1413 of external DN 1522.

[0107] Figure 16 Example 1600 shows content bootstrapping and some distributions outside of trusted DNs. For example, with Figure 15 In contrast, a trusted DN 1601 can exist, where the 5GMS AS 1612 can have at least one distribution server, such as "Distribution Server 1" 1602, and other servers (such as "Distribution Server 2" 1603 and a bootstrap server 1605) can be set up in an external DN 1622. In this collaboration, content bootstrap is provided by an external entity in the external DN 1622, guiding the UE 1401 to obtain content from multiple delivery networks, one of which is an MNO network with a trusted DN 1601.

[0108] And according to the combination Figure 16In an exemplary embodiment, (1) the MNO may provide one of the DNs, namely “Distribution Server 1” 1602, for delivering content to or from the UE 1401. The content may be delivered by other DNs outside the trusted DN 1601 of the MNO (such as “Distribution Server 2” 1603). AP 1413 has information about external DN 1622. According to the exemplary embodiment, the MNO may not be aware of the existence and nature of these networks. (2) Content bootstrapping server 1605 is also located in external DN 1622. (3) AP 1413 provides a list containing the DNs used by the MNO and the base URLs of the external DNs, and also contains information about the content bootstrapping service. (4) The client may use the MNO’s DN 1601 or the external network 1622 based on the response of content bootstrapping server 1605.

[0109] Figure 17 Example 1700 illustrates content bootstrapping outside the Trusted DN and distribution within the Trusted DN. For example, with Figure 15 In contrast, a trusted DN 1701 can exist, where the 5GMS AS 1712 can have at least two distribution servers, such as "Distribution Server 1" 1702 and "Distribution Server 2" 1703, and a bootstrap server 1705 can be set up in an external DN 1722. In this collaboration, content bootstrap is provided by an external entity in the external DN 1722, guiding the UE 1401 to obtain content from multiple delivery networks, one of which is an MNO network with a trusted DN 1701.

[0110] And according to the combination Figure 17 In an exemplary embodiment, (1) the MNO provides all DNs, such as the trusted DN 1701 of the MNO, "Distribution Server 1" 1702, for delivering content to or from the UE 1401. (2) the AP 1413 has information about the MNO DNs. (3)

[0111] The content bootstrapping server is located in external DN 1722. (4) AP 1413 provides a list containing the base URL of the DN for the MNO and information about the content bootstrapping service. (5) The client, namely client 1402 of UE 1401, can use the DN of the MNO based on the response of the content bootstrapping server 1705.

[0112] Figure 18 Example 1800 illustrates content bootstrapping within a Trusted DN and distribution both inside and outside the Trusted DN. For example, with Figure 15In contrast, a trusted DN 1801 can exist, where the 5GMS AS 1812 can have a distribution server (such as "Distribution Server 1" 1802) and a boot server 1805. "Distribution Server 2" 1803 can be located in an external DN 1822. In this collaboration, some DNs are located outside the MNO; however, content booting is provided by the MNO.

[0113] And according to the combination Figure 18 In an exemplary embodiment, (1) the MNO provides some DNs, such as “Distribution Server 1” 1802, for delivering content to or from the UE 1401, and for content bootstrapping via bootstrapping server 1805. (2) AP 1413 also provides the MNO with information about external DNs. (3) The MNO (e.g., via trusted DN 1801) manipulates a manifest to add the base URL of the DNs used by the MNO and information about the content bootstrapping service. (4) The client, i.e., client 1402 of the UE 1401, can use the MNO’s DNs (such as “Distribution Server 1” 1802) or external distributions (such as “Distribution Server 2” 1803) based on the response of the content bootstrapping server 1805. However, content bootstrapping is performed by the MNO.

[0114] Therefore, multiple methods for deploying content guidance services in 5G media delivery are provided, wherein the distribution server can be located inside or outside the MNO, and / or the content guidance server can be located inside or outside the MNO. In each case, the process of generating / manipulating the manifest is described, as well as the operation point where the client uses content guidance information to select and stream to or from the DN. Depending on the location of the DN and the location of the content guidance server, different information flows are required to effectively deploy the content guidance service.

[0115] Regarding general server-assisted and network-assisted streaming, embodiments of this application provide general server-assisted and network-assisted streaming scenarios for efficient content delivery using content bootstrapping mechanisms. These scenarios involve internal and external collaboration models, focusing on optimizing delivery paths, reducing latency, and improving bandwidth efficiency. References include ETSI TS 103 998 [ETSI-CS] for content bootstrapping in DASH environments to ensure compliance with industry standards.

[0116] Similar to that described for Example 1500, the MNO provides content bootstrapping between various distributions provided by the 5GMSd AS, and content bootstrapping and distribution within a trusted domain. The content bootstrapping server also resides within the trusted DN. And in such embodiments, (1) the MNO provides multiple 5GMSd AS instances to deliver content to or from the UE at reference point M4d. (2) the MNO also provides a content bootstrapping server as part of the 5GMSd AS. (3) the presentation manifest published by the 5GMSd AP at reference point M2d does not include any content bootstrapping information. The 5GMS system manipulates the manifest by adding a base URL and bootstrapping server information, and then provides it to the 5GMSd client at reference point M2d. And (4) during streaming, the UE makes a request to the content bootstrapping server based on the provided information. The content bootstrapping operation is performed within the MNO's 5GMS system and is opaque to the 5GMSd AP.

[0117] Similar to that described for Example 1600, content bootstrapping (such as content bootstrapping outside a trusted domain and hybrid content delivery inside and outside a trusted domain) is provided by an external entity in an external DN, bootstrapping the UE to obtain content from multiple delivery networks, one of which is the MNO's 5G system. And in such embodiments, (1) the MNO provides a 5GMSd AS for delivering content to or from the UE. The same content may also be obtained from other DNs outside the MNO's trusted DN. The 5GMSd AP has information about the external DNs. The MNO may not be aware of the existence and nature of these networks. (2) The content bootstrapping server is also located in the external DN. (3) The 5GMSd AP provides a presentation manifest at reference point M2d, which contains the base URLs for the 5GMSd AS for the MNO and the external DNs, and also contains information about the content bootstrapping service. (4) The 5GMSd client may use the MNO's 5GMSd AS at reference point M4d or use an external network, depending on the response of the content bootstrapping server.

[0118] Similar to that described for Example 1700, content guidance is provided, for example, content guidance outside the trusted domain and content delivery within the trusted domain provided by an external entity in the external DN, guiding the UE to retrieve content from multiple 5GMSd AS instances, all of which are deployed in the trusted DN of the MNO. And in such embodiments, (1) the MNO provides 5GMSd AS instances for delivering content to or from the UE. (2) the 5GMSd AP has information about the 5GMSd AS instances of the MNO. (3) the content guidance server is located in the external DN. (4) the AP provides a presentation manifest at reference point M2d, which contains the base URL for the 5GMSd AS instances of the MNO, as well as information about the external content guidance service. (5) the 5GMSd client uses one of the 5GMSd AS instances of the MNO at reference point M4d based on the response of the content guidance server.

[0119] Similar to that described for Example 1800, content guidance is provided, such as content guidance within a trusted domain provided by the MNO and content delivery within and outside the trusted domain. However, at least one DN exists outside the trusted DN. And in such embodiments, (1) the MNO provides some 5GMSd AS instances for delivering content to or from the UE. (2) the 5GMSd AP has information about the MNO 5GMSd AS instances. (3) the content guidance server is provided by the MNO. (4) the AP provides a presentation list at reference point M2d, which contains the base URLs of the 5GMSd AS instances for the MNO and the base URLs of the external content servers. (5) the 5GMSd client selects one or more external content servers at reference point M4d based on the response of the content guidance server.

[0120] According to an exemplary embodiment, Figures 15 to 18 Interfaces M1, M2, M4, M5, and M8 can also be considered as interfaces M1d, M2d, M4d, M5d, and M8d.

[0121] Therefore, an embodiment mapped to the existing 5G framework is provided, featuring enhancements to support content bootstrapping across different scenarios, such as trusted domains only: Within the trusted domain of the MNO, the architecture comprises multiple 5GMSd AS service locations / endpoints interconnected via reference points M4d and M8d. The content bootstrapping server dynamically allocates delivery paths. Bootstrapping is achieved by having DASH clients periodically access the content bootstrapping server to retrieve a boot manifest that indicates the availability and priority of service locations / endpoints to the player.

[0122] An embodiment mapped to the existing 5G framework is provided, featuring enhancements to support content bootstrapping across different scenarios, such as hybrid trusted domains and external domains: For scenarios where delivery spans trusted and external domains, the 5GMSd client interacts with the bootstrapping server through an interface outside the 3GPP scope. Inter-domain metadata exchange ensures appropriate selection between trusted 5GMSd AS endpoints / locations and external CDNs based on factors such as load balancing, geographic location, and service-level agreements.

[0123] Regarding advanced call procedures, embodiments of this application provide...

[0124] Advanced call flow involving multiple stages:

[0125] - Content discovery and manifest retrieval: The 5GMSd AP publishes a presentation manifest at M2d, which is expanded by the MNO to include bootstrap metadata (e.g., base URL, bootstrap logic).

[0126] - Bootstrapping Decisions and Content Requests: The 5GMSd client queries the bootstrapping server (via reference point M24d) for the optimal delivery path. The decision incorporates real-time factors such as network congestion, content cache location, and user QoS profiles.

[0127] - Content delivery: Based on the response from the bootstrap server, the 5GMSd client retrieves content from the selected 5GMSd AS endpoint / location (reference point M4d) or an external CDN.

[0128] - Adaptation and Monitoring: Delivery dynamically adapts to changing conditions, ensuring uninterrupted playback and meeting latency and throughput KPIs.

[0129] The collaborative scenarios provided in this application aim to address the challenges of integrating server-assisted and network-assisted streaming in hybrid environments, leveraging content bootstrapping to optimize delivery paths across trusted and external networks. Clause 5.19 of 3GPP TR26.804 18.2.0, concerning key issues related to media delivery from multiple service endpoints / locations, covers much of the considerations for introducing content bootstrapping into 5GMS.

[0130] The above-described technology can be implemented as computer software using computer-readable instructions and physically stored in at least one computer-readable medium, or implemented by at least one specially configured hardware processor. For example, Figure 19 A computer system 1900 is shown that is suitable for implementing certain embodiments of the disclosed subject matter.

[0131] Computer software can be written using any suitable machine code or computer language. This machine code or computer language may be processed by mechanisms such as assembly, compilation, and linking to generate code containing instructions. These instructions can be executed directly by the computer's central processing unit (CPU), graphics processing unit (GPU), or through interpretation, microcode execution, or other methods.

[0132] These instructions can be executed on various types of computers or their components, including personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, and so on.

[0133] Figure 19 The components of the computer system 1900 shown are merely exemplary and are not intended to imply any limitation on the scope or functionality of the computer software used to implement embodiments of the present invention. The configuration of the components should also not be construed as having any dependency or requirement on any component or combination of components shown in the exemplary embodiments of the computer system 1900.

[0134] Computer system 1900 may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from at least one human user via, for example, tactile input (e.g., key presses, swipes, data glove movements), audio input (e.g., voice, clapping sounds), visual input (e.g., gestures), or olfactory input (not depicted). Human-machine interface devices may also be used to acquire media that are not necessarily directly related to conscious human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from still image cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0135] Human-machine interface input devices may include at least one of the following (only one of each is depicted): keyboard 1901, mouse 1902, touchpad 1903, touch screen 1910, joystick 1905, microphone 1906, scanner 1908, and camera 1907.

[0136] Computer system 1900 may also include certain human-machine interface output devices. Such human-machine interface output devices can stimulate the senses of at least one human user through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (such as tactile feedback provided by touchscreen 1910 or joystick 1905, but there may also be tactile feedback devices that do not act as input devices), audio output devices (such as speakers 1909, headphones (not depicted)), visual output devices (such as screens 1910, including CRT screens, LCD screens, plasma screens, OLED screens, each screen may or may not have touchscreen input functionality, each screen may or may not have tactile feedback functionality—some of which are capable of outputting two-dimensional or more than three-dimensional visual output through means such as stereoscopic output; virtual reality glasses (not depicted), holographic displays, and smoke boxes (not depicted)), and printers (not depicted).

[0137] The computer system 1900 may also include user-accessible storage devices and their associated media, such as optical media or similar media including CD / DVD ROM / RW 1920 equipped with CD / DVD 1911, flash memory drives 1922, removable hard disk drives or solid state drives (SSDs) 1923, conventional magnetic media such as magnetic tapes and floppy disks (not depicted), and dedicated devices based on ROM / ASIC / PLD such as security dongles (not depicted).

[0138] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not cover transmission media, carrier waves, or other transient signals.

[0139] Computer system 1900 may also include an interface 1999 connected to at least one communication network 1998. Network 1998 may be, for example, a wireless network, a wired network, or an optical network. Network 1998 may also be a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), an in-vehicle and industrial network, a real-time network, a latency-tolerant network, etc. Examples of network 1998 include: local area networks such as Ethernet and wireless LANs; cellular networks including GSM, 3G, 4G, 5G, LTE, etc.; TV wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV; and in-vehicle and industrial networks including CAN buses, etc. Some networks 1998 typically require an external network interface adapter (e.g., a USB port of computer system 1900) attached to certain general-purpose data ports or peripheral buses (1950 and 1951); other networks are typically integrated into the core of computer system 1900 via a system bus described below (e.g., integrated into a PC computer system via an Ethernet interface, or integrated into a smartphone computer system via a cellular network interface). Using any of these networks (1998), computer system 1900 can communicate with other entities. This communication can be one-way receiving (e.g., broadcasting TV), one-way transmitting (e.g., CAN bus to certain CAN bus devices), or bidirectional, such as connecting to other computer systems using a local area or wide area digital network. Certain protocols and protocol stacks can be used on each of the networks and network interfaces described above.

[0140] The aforementioned human-machine interface device, user-accessible storage device, and network interface can be attached to the core 1940 of the computer system 1900.

[0141] Core 1940 may include at least one CPU 1941, GPU 1942, graphics adapter 1917, dedicated programmable processing unit in the form of a Field Programmable Gate Array (FPGA) 1943, hardware accelerators 1944 for certain tasks, etc. These devices, along with read-only memory (ROM) 1945, random-access memory (RAM) 1946, and internal high-capacity storage devices 1947 such as internal non-user-accessible hard drives and solid-state drives, can be connected via system bus 1948. In some computer systems, system bus 1948 can be accessed via at least one physical connector, allowing for expansion through additional CPUs, GPUs, etc. Peripheral devices can be directly attached to the core's system bus 1948 or via peripheral bus 1951. Peripheral bus architectures include PCI, USB, etc.

[0142] CPUs 1941, GPUs 1942, FPGAs 1943, and accelerators 1944 can execute certain instructions, which, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM 1945 or RAM 1946. Temporary data can also be stored in RAM 1946, while permanent data can be stored, for example, in an internal mass storage device 1947. Fast storage and retrieval of any of the storage devices can be implemented using a cache memory, which can be closely associated with at least one CPU 1941, GPU 1942, mass storage device 1947, ROM 1945, RAM 1946, etc.

[0143] The computer-readable medium may contain computer code for performing various computer-implemented operations. The medium and computer code may be designed and constructed specifically for the purposes of this invention, or may belong to a class well-known and available to those skilled in the art of computer software.

[0144] For example, but not as a limitation, the architecture corresponding to computer system 1900, particularly core 1940, can provide functionality generated when one or more processors (including CPUs, GPUs, FPGAs, accelerators, etc.) execute software embodied in at least one tangible computer-readable medium. Such a computer-readable medium can be a medium associated with certain storage devices (e.g., internal core mass storage 1947 or ROM 1945) of the non-transitory nature of core 1940, as described above as user-accessible mass storage devices. Software implementing various embodiments of the invention can be stored in such devices and executed by core 1940. Depending on specific needs, the computer-readable medium may include at least one storage device or chip. The software can cause core 1940, particularly its processors (including CPUs, GPUs, FPGAs, etc.), to execute specific processes or specific portions of specific processes described in this application, including defining data structures stored in RAM 1946 and modifying such data structures according to the processes defined by the software. Alternatively or as an alternative, the computer system may provide functionality generated by logic hard-wired or otherwise embodied in circuitry (e.g., accelerator 1944), which may operate in place of or in conjunction with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may cover logic, and vice versa. Where appropriate, references to computer-readable media may cover circuitry (e.g., an IC) storing software for execution, circuitry embodying logic for execution, or both. This invention covers any suitable combination of hardware and software.

[0145] While several exemplary embodiments have been described in this invention, modifications, substitutions, and various alternative equivalents exist that fall within the scope of this invention. Therefore, it should be understood that those skilled in the art will be able to design numerous systems and methods that, while not explicitly stated or described herein, embody the principles of this invention and are therefore within its spirit and scope.

Claims

1. A video decoding method, executed by at least one processor, characterized in that, The method includes: At least one 5G media downlink stream 5GMSd instance is generated by the Trusted Distribution Network (DN) of the mobile network operator (MNO). The at least one 5GMSd instance is configured to deliver at least a portion of the content to a user equipment (UE). The UE includes a 5GMSd-aware application and a 5GMSd client. The at least a portion of the content includes video data. The trusted DN obtains a presentation list published by the 5GMSd application provider AP of an external DN, which is separate from both the trusted DN and the UE; and The content is streamed to the UE based on the presentation list, and the content is decoded at the UE.

2. The method according to claim 1, characterized in that, The presentation list is obtained by the trusted DN from the 5GMSd AP via the M2d application programming interface API.

3. The method according to claim 1, characterized in that, The streaming of the content includes: streaming at least a portion of the content from the 5GMSd AS to the UE via the M4d application programming interface API.

4. The method according to claim 1, characterized in that, The MNO provides multiple 5GMSd instances, including at least one of the 5GMSd instances, which are configured to jointly deliver the content to the UE.

5. The method according to claim 1, characterized in that, The external DN provides at least one other 5GMSd instance, which is configured to deliver the content to the UE together with the at least one 5GMSd instance of the trusted DN.

6. The method according to claim 1, characterized in that, The content streaming to the UE is guided by the guidance server of the trusted DN.

7. The method according to claim 1, characterized in that, The content delivery to the UE is guided by the boot server of the external DN.

8. A video encoding method, executed by at least one processor, characterized in that, The method includes: At least one 5G media downlink stream 5GMSd instance is generated by the Trusted Distribution Network (DN) of the mobile network operator (MNO). The at least one 5GMSd instance is configured to deliver at least a portion of the content to a user equipment (UE). The UE includes a 5GMSd-aware application and a 5GMSd client. The at least a portion of the content includes video data. The trusted DN obtains a presentation list published by the 5GMSd application provider AP of an external DN, which is separate from both the trusted DN and the UE; and The content is encoded and streamed to the UE based on the presentation list.

9. The method according to claim 8, characterized in that, The presentation list is obtained by the trusted DN from the 5GMSd AP via the M2d application programming interface API.

10. The method according to claim 8, characterized in that, The streaming of the content includes: streaming at least a portion of the content from the 5GMSd AS to the UE via the M4d API.

11. The method according to claim 8, characterized in that, The MNO provides multiple 5GMSd instances, including at least one of the 5GMSd instances, which are configured to jointly deliver the content to the UE.

12. The method according to claim 8, characterized in that, The external DN provides at least one other 5GMSd instance, which is configured to deliver the content to the UE together with the at least one 5GMSd instance of the trusted DN.

13. The method according to claim 8, characterized in that, The content streaming to the UE is guided by the guidance server of the trusted DN.

14. The method according to claim 8, characterized in that, The content delivery to the UE is guided by the boot server of the external DN.

15. A method for processing visual media data, executed by at least one processor, characterized in that, The method includes: At least one 5G media downlink stream 5GMSd instance is generated by the Trusted Distribution Network (DN) of the mobile network operator (MNO). The at least one 5GMSd instance is configured to deliver at least a portion of the content to a user equipment (UE). The UE includes a 5GMSd-aware application and a 5GMSd client. The at least a portion of the content includes video data. The trusted DN obtains a presentation list published by the 5GMSd application provider AP of an external DN, which is separate from both the trusted DN and the UE; and According to the format rules, a conversion is performed between the visual media file of the content and the bitstream of the visual media data of the content, and based on the conversion, at least one of streaming the content to the UE and playing the content by the UE is performed.

16. The method according to claim 15, characterized in that, The presentation list is obtained by the trusted DN from the 5GMSd AP via the M2d application programming interface API.

17. The method according to claim 15, wherein streaming the content in this application comprises: At least a portion of the content is streamed from the 5GMSd AS to the UE via the M4d API.

18. The method according to claim 15, characterized in that, The MNO provides multiple 5GMSd instances, including at least one of the 5GMSd instances, which are configured to jointly deliver the content to the UE.

19. The method according to claim 15, characterized in that, The external DN provides at least one other 5GMSd instance, which is configured to deliver the content to the UE together with the at least one 5GMSd instance of the trusted DN.

20. The method according to claim 19, characterized in that, The content streaming to the UE is guided by a guide server of either the trusted DN or the external DN.

Images