Multi-modality support in extended reality

Abstract

Embodiment provide User equipment, for processing multi-modal data, the multi modal data including at least two data flows, wherein the user equipment is configured to construct at least one multi-modal data block, each of the at least one multi-modal data blocks comprising a plurality of data packets of the at least two data flows, the plurality of data packets comprising a reference data packet of a first data flow of the at least two data flows and one or more secondary data packets of at least a second data flow of the at least two data flows, wherein the secondary data packets are related to the reference data packet.

Description

[0001] 24030 1

[0002] Multi-Modality Support in Extended Reality

[0003] Description

[0004] Embodiments of the present application relate to the field of wireless communication, and more specifically, to a transmission and / or reception of multi-modal data. Some embodiments relate to multi-modality support in extended reality.

[0005] Fig. 1 is a schematic representation of an example of a terrestrial wireless network 100 including, as is shown in Fig. 1(a), a core network 102 and one or more radio access networks (RANs) RAN1 , RAN2, ... RANN. Fig. 1 (b) is a schematic representation of an example of a radio access network RANn that may include one or more base stations (BSs) gNB1 to gNB5, each serving a specific area surrounding the base station schematically represented by respective cells 1061 to 1065. The base stations are provided to serve users within a cell. The term base station, BS, refers to a next generation node B (gNB) in 5G networks, an evolved node B (eNB) in UMTS / LTE / LTE-A / LTE-A Pro, or just a BS in other mobile communication standards. A user may be a stationary device or a mobile device. The wireless communication system may also be accessed by mobile or stationary Internet of Things (loT) devices which connect to a base station or to a user. The mobile devices or the loT devices may include physical devices, ground based vehicles, such as robots or cars, aerial vehicles, such as manned or unmanned aerial vehicles (UAVs), the latter also referred to as drones, buildings and other items or devices having embedded therein electronics, software, sensors, actuators, or the like as well as network connectivity that enables these devices to collect and exchange data across an existing network infrastructure. Fig. 1 (b) shows an exemplary view of five cells, however, the RANn may include more or less such cells, and RANn may also include only one base station. Fig. 1(b) shows two users UE1 and UE2, also referred to as user equipment, UE, that are in cell 1062 and that are served by base station gNB2. Another user UE3 is shown in cell 1064 which is served by base station gNB4. The arrows 1081 , 1082 and 1083 schematically represent uplink / downlink connections for transmitting data from a user UE1 , UE2 and UE3 to the base stations gNB2, gNB4 or for transmitting data from the base stations gNB2, gNB4 to the users UE1 , UE2, UE3. Further, Fig. 1(b) shows two loT devices 1101 and 1102 in cell 1064, which may be stationary or mobile devices. The loT device 1101 accesses the wireless communication system via the base station gNB4 to receive and transmit data as schematically represented by arrow 1121. The loT device 1102 accesses the wireless communication system via the user UE3 as is schematically represented by arrow 1122. The

[0006] FH240906PEP-2025313838. DOCX PCT 24030 2 respective base station gNB1 to gNB5 may be connected to the core network 102, e.g., via the S1 interface, via respective backhaul links 1141 to 1145, which are schematically represented in Fig. 1(b) by the arrows pointing to “core”. The core network 102 may be connected to one or more external networks. Further, some or all of the respective base station gNB1 to gNB5 may connected, e.g., via the S1 or X2 interface or the XN interface in NR, with each other via respective backhaul links 1161 to 1165, which are schematically represented in Fig. 1 (b) by the arrows pointing to “gNBs”.

[0007] For data transmission a physical resource grid may be used. The physical resource grid may comprise a set of resource elements (REs) to which various physical channels and physical signals are mapped. For example, the physical channels may include the physical downlink, uplink and sidelink shared channels (PDSCH, PLISCH, PSSCH) carrying user specific data, also referred to as downlink, uplink and sidelink payload data, the physical broadcast channel (PBCH) carrying for example a master information block (MIB), the physical downlink shared channel (PDSCH) carrying for example a system information block (SIB), the physical downlink, uplink and sidelink control channels (PDCCH, PLICCH, PSSCH) carrying for example the downlink control information (DCI), the uplink control information (UCI) and the sidelink control information (SCI). For the uplink, the physical channels, or more precisely the transport channels according to 3GPP, may further include the physical random access channel (PRACH or RACH) used by UEs for accessing the network once a UE is synchronized and has obtained the MIB and SIB. The physical signals may comprise reference signals or symbols (RS), synchronization signals and the like. The resource grid may comprise a frame or radio frame having a certain duration in the time domain and having a given bandwidth in the frequency domain. The frame may have a certain number of subframes of a predefined length, e.g., 1 ms. Each subframe may include one or more slots of 12 or 14 orthogonal frequency-division multiplexing (OFDM) symbols depending on the cyclic prefix (CP) length. All OFDM symbols may be used for downlink (DL) or uplink (UL) or only a subset, e.g., when utilizing shortened transmission time intervals (sTTI) or a mini-slot / non-slot-based frame structure comprising just a few OFDM symbols.

[0008] The wireless communication system may be any single-tone or multicarrier system using frequency-division multiplexing, like the OFDM system, the orthogonal frequency-division multiple access (OFDMA) system, or any other IFFT-based signal with or without CP, e.g., DFT-s-OFDM. Other waveforms, like non-orthogonal waveforms for multiple access, e.g., filter-bank multicarrier (FBMC), generalized frequency division multiplexing (GFDM) or universal filtered multi carrier (LIFMC), may be used. The wireless communication system may

[0009] FH240906PEP-2025313838. DOCX PCT 24030 3 operate, e.g., in accordance with the LTE-Advanced pro standard or the NR (5G), New Radio, standard.

[0010] The wireless network or communication system depicted in Fig. 1 may by a heterogeneous network having distinct overlaid networks, e.g., a network of macro cells with each macro cell including a macro base station, like base station gNB1 to gNB5, and a network of small cell base stations (not shown in Fig. 1), like femto or pico base stations.

[0011] In addition to the above described terrestrial wireless network also non-terrestrial wireless communication networks exist including spaceborne transceivers, like satellites, and / or airborne transceivers, like unmanned aircraft systems. The non-terrestrial wireless communication network or system may operate in a similar way as the terrestrial system described above with reference to Fig. 1 , for example in accordance with the LTE-Advanced Pro standard or the NR (5G), new radio, standard.

[0012] In mobile communication networks, for example in a network like that described above with reference to Fig. 1 , like an LTE or 5G / NR network, there may be UEs that communicate directly with each other over one or more sidelink (SL) channels, e.g., using the PC5 interface. UEs that communicate directly with each other over the sidelink may include vehicles communicating directly with other vehicles (V2V communication), vehicles communicating with other entities of the wireless communication network (V2X communication), for example roadside entities, like traffic lights, traffic signs, or pedestrians. Other UEs may not be vehicular related UEs and may comprise any of the above-mentioned devices. Such devices may also communicate directly with each other (D2D communication) using the SL channels.

[0013] In a communication system as described above, an XR service application, for example, running on a UE, may generate multi-modal data (e.g., video, audio, pose) around the same time. The related different data flows carry information referring to the same context and are thus inter-dependent.

[0014] However, so far, the different data flows are handled independently. This may result in missing synchronization of different modal data as these may have, e.g., different priorities, QoS and synchronization requirements.

[0015] Therefore, there is the need for improvements or enhancements with respect to processing and / or transmission of multi-modal data.

[0016] FH240906PEP-2025313838. DOCX PCT 24030 4

[0017] It is noted that the information in the above section is only for enhancing the understanding of the background of the invention and therefore it may contain information that does not form prior art and is already known to a person of ordinary skill in the art.

[0018] Embodiments of the present invention are described herein making reference to the appended drawings.

[0019] Fig. 1 shows a schematic representation of an example of a wireless communication system;

[0020] Fig. 2 is a schematic representation of a group of pictures and the relationship between the frames of the group of pictures, i.e. the relationship between l-frames and P- frames;

[0021] Fig. 3 is a schematic representation of multi-modal data generated by two XR service applications;

[0022] Fig. 4 is a schematic representation of a wireless communication system comprising a transceiver, like a base station or a relay, and a plurality of communication devices, like UEs, according to an embodiment,

[0023] Fig. 5 is a schematic representation of multi-modal data comprising a plurality of data flows, each data flow comprising a plurality of data packets, as well as an allocation of data packets of different data flows to two different multi-modal data blocks;

[0024] Fig. 6 is a schematic representation of multi-modal data comprising a plurality of data flows, each data flow comprising a plurality of data packets, as well as an allocation of data packets of different data flows to a multi-modal data bloc;

[0025] Fig. 7 is a schematic representation of a first step of a discard operation applied based on the multi-modal data blocks shown in Fig. 5,

[0026] Fig. 8 is a schematic representation of a second step of a discard operation applied based on the multi-modal data blocks shown in Fig. 5,

[0027] FH240906PEP-2025313838. DOCX PCT 24030 5

[0028] Fig. 9 is a schematic representation of a third step of a discard operation applied based on the multi-modal data blocks shown in Fig. 5,

[0029] Fig. 10 is a schematic representation of a fourth step of a discard operation applied based on the multi-modal data blocks shown in Fig. 5,

[0030] Fig. 11 is a schematic representation of an uplink QoS model with an extension for processing / transmitting multi-modal data;

[0031] Fig. 12 is a schematic representation of a downlink QoS model with an extension for processing / transmitting multi-modal data; and

[0032] Fig. 13 illustrates an example of a computer system on which units or modules as well as the steps of the methods described in accordance with the inventive approach may execute.

[0033] Equal or equivalent elements or elements with equal or equivalent functionality are denoted in the following description by equal or equivalent reference numerals.

[0034] In the following description, a plurality of details is set forth to provide a more thorough explanation of embodiments of the present invention. However, it will be apparent to one skilled in the art that embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail in order to avoid obscuring embodiments of the present invention. In addition, features of the different embodiments described hereinafter may be combined with each other, unless specifically noted otherwise.

[0035] Before embodiments of the present invention are described in further detail, some background information on multi-modal data (e.g., generated by an XR application) is provided.

[0036] 1. _ Multi-Modality Traffic Flow

[0037] A multi-modal service can be defined as a communication service that consists of several data flows that relate to each other and that are subject to application coordination. This typically includes video, audio, and viewer pose information, whose most relevant characteristics we will describe in this Section.

[0038] FH240906PEP-2025313838. DOCX PCT 24030 6

[0039] Note that other data flows, e.g., haptic, sensor measurements, etc., and can also be considered in multi-modal services.

[0040] Note that concepts described herein are agnostic of the nature of the data flows of the multimodal service.

[0041] 1.1 Video

[0042] The frame rate for Extended Reality (XR) video varies from 15 frames per second (fps) up to 90 or even 120 fps, with a typical minimum of 60 fps for Virtual Reality (VR). The motion-to- photon latency should be less than 20 ms, with 10 ms being given as a goal. Regarding the bit rates, between 10 and 200 Mbps can be expected for XR depending on frame rate, resolution, and codec efficiency.

[0043] When the video encoding scheme is based on the group of picture (GOP) model, a single video frame is either an l-frame or a P-frame:

[0044] • An l-frame is a complete image which can be decoded independently, l-frames are transmitted every K frames, where K is the GOP size (i.e., every GOP),

[0045] • A P-frame in the same GOP only shows the changes in the image from the previous frame and cannot be decoded without the help of the l-frame and of the previous P- frames.

[0046] One video frame arrives at a time and corresponds to a PDU Set. Generally, the first frame of a GOP is an l-frame, and the subsequent frames are P-frames. The relationship between GOP, l-frames and P-frames is illustrated in Fig. 2, which shows a schematic representation of a GOP 600 and the relationship between the frames of a GOP 600, i.e. the relationship between l-frames 602 and P-frames 604.

[0047] In other words, a dependency exists among the frames (i.e., among the corresponding PDU Sets) during the transmission, which is meaningful to consider.

[0048] Note that such inter-dependencies exist only for video streams. According to the traffic models defined in [3], they do not exist for the audio, pose information, and haptic streams.

[0049] Note that the traffic characteristics and requirements used herein “can only be used as a baseline when specific examples for XR traffic characteristics are needed - bearing in mind that they are not universally applicable for all XR applications” [5], For example, “low-latency

[0050] FH240906PEP-2025313838. DOCX PCT 24030 7

[0051] XR and cloud gaming video services such as Split-Rendering or Cloud Gaming typically would not use the traditional coding structure with a fixed Group of-Picture (GOP). In addition, the field of low-latency video delivery is undergoing heavy innovation and new coding methods may be established”.

[0052] 1.2 Audio

[0053] “[...], due to the relatively slower speed of sound compared to that of light, it is natural that users are more accustomed to, and therefore tolerant of, sound being relatively delayed with respect to the video component than sound being relatively in advance of the video component. Recent studies have led to recommendations of an accuracy of between 15 ms (audio delayed) and 5 ms (audio advanced) for the synchronization, with recommended absolute limits of 60 ms (audio delayed) and 40 ms (audio advanced) for broadcast video” [5],

[0054] 1.3 Pose

[0055] The XR Viewer Pose describes the state (i.e. , the position and orientation in space) of a viewer of the XR scene as tracked by the XR device, to which is also assigned a time stamp.

[0056] To maintain a reliable registration of the virtual world with the real world as well as to ensure accurate tracking of the XR Viewer pose, XR applications require highly accurate, low-latency tracking of the device at about 1 kHz sampling frequency. The size of a XR Viewer Pose typically results in packets of size in the range of 30-100 bytes, such that the generated data is around several hundred kbit / s if delivered over the network, with latency requirements in the range of 10-20 ms [5], [6],

[0057] However, it can be assumed that sending one pose packet aligned with the frame rate of the rendered video may be sufficient, e.g., at 60 Hz.

[0058] 1 .4 Application level awareness

[0059] The XR service application generates multi-modal data (i.e., originating from different media sources, e.g., video, audio, pose, and haptic) at around the same time. The different data streams carry information referring to the same context and are thus inter-dependent. This is particularly important when the synchronization threshold between two or more modalities is less than the latency requirement for the XR service application.

[0060] FH240906PEP-2025313838. DOCX PCT 24030 8

[0061] The coordinated transmission of these data streams translates into tight synchronization requirements [1] to avoid having a negative impact on the user experience (i.e., viewers detecting lack of synchronization between the different medias).

[0062] These types of synchronization are typically handled at the level of the XR service application. The end-to-end (E2E) synchronization of different traffic flows belonging to the same multimodal service is possible, e.g., see [7], by means of:

[0063] • Timestamps, both Network Time Protocol (NTP) and Real-time Transport Protocol (RTP).

[0064] • Receiver Reports.

[0065] Therefore, it can be concluded that awareness of the coordinated flows at the lower layers can also be beneficial to (improve) the user experience.

[0066] 1 .5 Multi-modal service ID (QoS flow level)

[0067] Fig. 3 shows a schematic representation of multi-modal data generated by two XR service applications.

[0068] A first XR service application may generate first multi-modal data 402. The first multi-modal data 402 can comprise at least two data flows, such as a first data flow 404, a second data flow 406 and a third data flow 408. Each of the data flows 404-408 can comprise a plurality of data packets, such as a plurality of payload data sets (e.g., PDU sets). For example, a first data flow 404 can comprise first data packets 410_1-410_3, where the second data flow 406 can comprise second data packets 412_1 -412_5, where the third data flow 408 can comprise third data packets 414_1-414_20. Thereby, note that in Fig. 3, only a portion / extract of the data flows 404-408 is shown. Further, a first multi-modal service ID (MMSID) associated with the first multi-modal data 402 is indicated in Fig. 3.

[0069] Similarly, a second XR service application may generate second multi-modal data 422. The second multi-modal data 422 can comprise at least two data flows, such as a first data flow 424, a second data flow 426 and a third data flow 428. Each of the data flows 424-428 can comprise a plurality of data packets, such as a plurality of payload data sets (e.g., PDU sets). For example, a first data flow 424 can comprise first data packets 430_1-430_3, where the second data flow 426 can comprise second data packets 432_1-432_5, where the third data flow 428 can comprise third data packets 434_1-434_20. Thereby, note that in Fig. 3, only a

[0070] FH240906PEP-2025313838. DOCX PCT 24030 9 portion / extract of the data flows 424-428 is shown. Further, a second multi-modal service ID (MMSID) associated with the second multi-modal data 422 is indicated in Fig. 3.

[0071] As shown in Fig. 3, the multi-modal service ID (MMSID) operates at the Quality of Service (QoS) flow level. The MMSID only indicates that some QoS flows belong to the same multimodal service. No more, no less. However, it does not provide any inter-dependency information with finer granularity, e.g., between the PDU Sets within and / or between these QoS flows.

[0072] Therefore, it can be concluded that providing the MMSID alone is sufficient in order to enable multi-modal awareness.

[0073] 1 .6 PDU Set discard operation

[0074] The Packet Data Convergence Protocol (PDCP) is a layer-2 sub-layer which provides a number of services for the processing of user and control plane data in transmission and in reception [4], This includes, but is not limited to, header compression and decompression, ciphering and deciphering, integrity protection and integrity verification, timer based service data unit (SDU) discard, PDU Set discard, duplication, reordering and in-order delivery, out- of-order delivery, duplicate discarding.

[0075] The SDU discard procedure consists in discarding a PDCP SDU when:

[0076] • an associated timer expires, or

[0077] • the receiving PDCP entity confirms the successful delivery of the PDCP SDU via a PDCP status report. This includes a bitmap indicating:

[0078] ’O’, for all PDCP SDUs that have not been received,

[0079] 'T for all PDCP SDUs that have been received.

[0080] On the other hand, the PDU Set discard operation consists in discarding a whole PDU Set when a least one of the corresponding PDCP SDU(s) is discarded by the discard timer. However, when a PDCP SDU is discarded by the status report, only this PDCP SDU is discarded [4, §5.3], Furthermore, two enhancements have been introduced in 3GPP Rel-18 XR [5], i.e.

[0081] Discard based on PDU Set: When the PDU Set Integrated Handling Indication (PSIHI) is set for a QoS flow and network configures a user equipment (UE) to perform PDU Set discard, as soon as one PDU of a PDU Set is known to be lost, the remaining PDUs of that PDU Set can be considered as no longer needed by the application and may be

[0082] FH240906PEP-2025313838. DOCX PCT 24030 10 subject to discard operation at the transmitter to free up radio resources. In other words, when configured, the UE discards all packets in a PDU Set when one PDU belonging to this PDU Set is discarded due to the expiry of its PDCP discard timer.

[0083] Discard based on PDU Set importance (PSI): In case of congestion, when configured, the UE is requested to use a shorter PDCP discard timer for low importance SDUs, which yields earlier discard of low importance SDUs or PDU Sets.

[0084] Therefore, it can be concluded that in Rel-18, the PDU set information contains only the information about a single PDU Set and does not include any inter-PDU Set dependency information. The Radio Access Network (RAN) can thus discard only one PDU Set at a time and cannot discard all the correlated PDU Sets.

[0085] Thereby, the inter-PDU set dependency information can be within one flow (e.g., between I- frames and P-frames for video streams) or across the multi-modal flows.

[0086] Further, it can be concluded that, if the UE or the RAN is aware of the inter-PDU Set dependency (within and across multi-modal flows), it can avoid sending over the air PDU Sets that are not considered necessary by the XR application service when one of the critical PDU Set is not successfully transmitted.

[0087] 2. _ Embodiments

[0088] Subsequently, embodiments are described that allow for improving a processing and / or transmission of multi-modal data.

[0089] Embodiments of the present invention may be implemented in a wireless communication system or network as depicted in Fig. 1 including a transceiver, like a base station, gNB, or relay, and a plurality of communication devices, like user equipment’s, UEs. Fig. 4 is a schematic representation of a wireless communication system comprising a transceiver 200, like a base station, and a plurality of communication devices 202_1 to 202_n, like UEs. The UEs might communicate directly with each other via a wireless communication link or channel 203, like a radio link (e.g., using the PC5 interface (sidelink)). Further, the transceiver and the UEs 202 might communicate via a wireless communication link or channel 204, like a radio link (e.g., using the Uu interface). The transceiver 200 might include one or more antennas ANT or an antenna array having a plurality of antenna elements, a signal processor 200a and a transceiver unit 200b. The UEs 202 might include one or more antennas ANT or an antenna array having a plurality of antennas, a processor 202a_1 to 202a_n, and a transceiver (e.g.,

[0090] FH240906PEP-2025313838. DOCX PCT 24030 11 receiver and / or transmitter) unit 202b_1 to 202b_n. The base station 200 and / or the one or more UEs 202 may operate in accordance with the inventive teachings described herein.

[0091] Embodiments provide a user equipment, for processing [e.g., and / or transmitting] multi-modal data, the multi modal data including at least two data flows, wherein the user equipment is configured to construct at least one multi-modal data block [or multi-modal data structure], each of the at least one multi-modal data blocks comprising a plurality of data packets [e.g., payload data sets] of [e.g., each of] the at least two data flows, the plurality of data packets [e.g., payload data sets] comprising a reference data packet [e.g., payload data set [e.g., reference PDU set]] of a first data flow of the at least two data flows and one or more secondary data packets [e.g., payload data sets] of at least a second data flow of the at least two data flows, wherein the secondary data packets [e.g., payload data sets] are related to [e.g., or dependent form] the reference data packet [e.g., payload data set].]

[0092] For example, each of the at least one multi-modal data blocks comprising a plurality of data packets [e.g., payload data sets] across the at least two data flows.

[0093] For example, each sequence of data packets within each data flow may extend over different time periods.

[0094] In embodiments, the at least two different data flows are associated with different types of information [e.g., video, audio, pose].

[0095] In embodiments, each of the at least two different data flows comprises a sequence of data packets [e.g., payload data sets [e.g., PDU sets]].

[0096] For example, the first data flow comprises a sequence of first data packets [e.g., payload data sets], where the second data flow comprises a sequence of second data packets [e.g., payload data sets].

[0097] In embodiments, the reference data packet [e.g., payload data set] is a proper subset of the sequence of data packets [e.g., payload data sets] of the first data flow, wherein the one or more secondary data packets [e.g., payload data sets] are a proper subset of the sequence of data packets [e.g., payload data sets] of the second data flow.

[0098] In embodiments, the one or more secondary data packets [e.g., payload data sets] are related to the reference data packet [e.g., payload data set] based on timing or context.

[0099] FH240906PEP-2025313838. DOCX PCT 24030 12

[0100] In embodiments, the user equipment is configured to construct the at least one multi-modal data block based on a timing information and / or a context information [e.g., provided by an extended reality application [e.g., executed by the user equipment]].

[0101] For example, the user equipment can be configured to construct each of the at least one multimodal data block by selecting the plurality of data packets out of a sequence of data packets of a first data flow and of a sequence of data packets of a second data flow based on the timing information and / or the context information.

[0102] In embodiments, the UE is configured to execute an extended reality application which provides the timing information and / or the context information about the multi-modal data to the lower layers. The UE is then configured to construct the multi-modal data block based on the timing information and / or the context information.

[0103] For example, the timing information can describe multi-modal events being presented to the user within the same predetermined time interval.

[0104] For example, the context information can include multi-modal events being presented to the user relating to the same presented event.

[0105] In embodiments, the user equipment comprises an entity configured to construct the at least one multi-modal data block.

[0106] For example, the entity can be media unit classification entity. The media unit classification entity can reside in the application layer or any lower layer, e.g., SDAP layer.

[0107] In embodiments, the reference data packet [e.g., payload data set] and the one or more secondary data packets [e.g., payload data sets] require a coordinated processing at a receiver side [e.g., in order to maintain synchronization and / or QoS coordination].

[0108] In embodiments, the user equipment is configured to transmit the at least one multi-modal data block.

[0109] In embodiments, the user equipment is configured to transmit at least one signaling information, the signaling information signaling the at least one multi-modal data block,

[0110] FH240906PEP-2025313838. DOCX PCT 24030 13

[0111] In embodiments, the signaling information is associated with the multi-modal data block.

[0112] In embodiments, the signaling information comprises [e.g., for each data packet [e.g., payload data set]] at least one out of a multi-modal service identification, MMSID, a multi-modal data block identification, MMDB-ID.

[0113] In embodiments, the signaling information comprises for each data packet [e.g., payload data set] at least one out of a data packet type indicator [e.g., reference or secondary payload data], a data flow dependency indicator [e.g., important [e.g., l-frame] or not-important [e.g., P-frame],

[0114] In embodiments, the plurality of data packets are one out of a plurality of payload data sets, a plurality of radio link control packets.

[0115] In embodiments, the user equipment comprises a data packet classification entity [e.g., PDU set classification entity] configured to generate the signaling information.

[0116] In embodiments, the user equipment is configured, in case that a discarding criterion of the reference data packet [e.g., payload data set] of a respective multi-modal data block is fulfilled, to discard at least one [e.g. all] of the other data packets [e.g., payload data sets] of the respective multi-modal data block.

[0117] For example, in case that a discarding criterion of the reference data packet [e.g., payload data set] of a multi-modal data block is fulfilled: (1) one or more other data packets of the respective multi-modal data block can be discarded, or (2) all other data packets belonging to one or more specific other data flows can be discarded, or (3) all other data packets of the respective multimodal data block can be cancelled.

[0118] For example, a reference flow / dataset [e.g., video] may be discardable, but a secondary flow / dataset [e.g., audio] of the multi-model flow can be maintained. In some embodiments, an additional configuration / signaling of the discarding mechanism can be used, e.g., indicating which of the flows to discard in case the discarding criterion is fulfilled.

[0119] FH240906PEP-2025313838. DOCX PCT 24030 14

[0120] In embodiments, the user equipment is configured to discard the signaling information associated with the multi-modal data block.

[0121] In embodiments, in case of a reception of an information [e.g., report or signaling or message] signaling [e.g., confirming] a successful delivery of at least the reference data packet [e.g., payload data set], the user equipment is configured to discard the at least one data packet indicated in the signaled information, or wherein the user equipment is configured to discard all the data packets of the multi-modal data block in case of the expiration of a discarding timer [e.g., associated with the reference data packet or with the entire multimodal data block].

[0122] For example, the discarding criterion can be fulfilled in case of a reception of a discard information [e.g., report or signaling or message] signaling [e.g., confirming] a successful delivery or reception of the respective multimodal data block or signaling [e.g., confirming] a successful delivery or reception of the reference data packet [e.g., payload data set].

[0123] For example, the discarding criterion can be fulfilled in case of an expiration of a discarding timer associated with the multi-modal data block and / or the reference data packet.]

[0124] In embodiments, the discard information (or signaled information) can be a bitmap with one bit per data packet, e.g., PDU or PDU Set.

[0125] In embodiments, in case that a data packet belonging to the respective multi-modal data block is received [e.g., received from a higher layer [e.g., higher than a layer on which the discarding is performed [e.g., PDCP layer]]] after at least one [e.g., all] of the other data packets of the respective multi modal data block was discarded, said data packet is also discarded, wherein, in case that a data packet belonging to the respective multi-modal data block was already processed [e.g., and transmitted to a lower layer [e.g., lower than a layer on which the discarding is performed [e.g., PDCP layer]]] before at least one [e.g., all] of the other data packets of the respective multi modal data block was discarded, said data packet is also discarded [e.g., in response to a respective signaling from where the discarding is performed to lower layers [e.g., from PDCP to lower layers]].

[0126] In embodiments, the signaling information comprises a discarding indicator indicating whether the discarding of at least one [e.g., all] of the data packets of the respective multi-modal data block should be performed in case that the discarding criterion is fulfilled, wherein the user

[0127] FH240906PEP-2025313838. DOCX PCT 24030 15 equipment is configured to discard at least one [e.g., all] of the data packets [e.g., payload data sets] of the respective multi-modal data block only in case that a discarding indicator indicates that the discarding of the at least one [e.g., all] of the data packets of the respective multimodal data block should be performed in case that the discarding criterion is fulfilled.

[0128] In embodiments, the discarding indicator is provided as part of a QoS profile.

[0129] In embodiments, the user equipment is configured to receive the discarding indicator from a core network entity.

[0130] In embodiments, in case that a data packet of a first multi-modal data block is discarded and a data packet of a second multi-modal data block [e.g., of the same data flow] is related [or depends from] the data packet of the first multi-modal data block, then also the data packet of the second multi-modal data block is discarded.

[0131] In embodiments, the signaling information comprises a data flow dependency indicator indicating whether the data packet of the second multi-modal data block is related to the data packet of the first multi-modal data block, wherein the user equipment is configured to discard the data packet of the second multi-modal data block only in case that the data flow dependency indicator indicates that the data packet of the second multi-modal data block is related to the data packet of the first multi-modal data block.

[0132] In embodiments, the user equipment is configured to discard all other data packets of the second multi-modal data block in case that a discarding indicator indicates that the discarding of all other data packets of the second multi-modal data block should be performed in case that the data packet is discarded.

[0133] Further embodiments provide a base station, for processing [e.g., and / or transmitting] multimodal data, the multi modal data including at least two data flows, [e.g., each of the at least two data flows comprising a sequence of data packets [e.g., payload data sets [e.g., PDU sets]]], wherein the base station is configured to construct at least one multi-modal data block or multi-modal data structure], each of the at least one multi-modal data blocks comprising a plurality of data packets [e.g., payload data sets] of [e.g., each of] the at least two data flows, the plurality of data packets [e.g., payload data sets] comprising a reference data packet [e.g., payload data set [e.g., reference PDU set]] of a first data flow of the at least two data flows and one or more secondary data packets [e.g., payload data sets] of at least a second data flow of

[0134] FH240906PEP-2025313838. DOCX PCT 24030 16 the at least two data flows, wherein the secondary data packets [e.g., payload data sets] are related to [e.g., or dependent form] the reference data packet [e.g., payload data set].

[0135] For example, each of the at least one multi-modal data blocks comprising a plurality of data packets [e.g., payload data sets] across the at least two data flows.

[0136] For example, each sequence of data packets within each data flow may extend over different time periods.

[0137] In embodiments, the at least two different data flows are associated with different types of information [e.g., video, audio, pose].

[0138] In embodiments, each of the at least two different data flows comprises a sequence of data packets [e.g., payload data sets [e.g., PDU sets]].

[0139] For example, the first data flow comprises a sequence of first data packets [e.g., payload data sets], where the second data flow comprises a sequence of second data packets [e.g., payload data sets].

[0140] In embodiments, the reference data packet [e.g., payload data set] is a proper subset of the sequence of data packets [e.g., payload data sets] of the first data flow, wherein the one or more secondary data packets [e.g., payload data sets] are a proper subset of the sequence of data packets [e.g., payload data sets] of the second data flow.

[0141] In embodiments, the one or more secondary data packets [e.g., payload data sets] are related to the reference data packet [e.g., payload data set] based on timing or context.

[0142] In embodiments, the base station is configured to construct the at least one multi-modal data block based on a timing and / or context information [e.g., provided by an extended reality application [e.g., executed by the base station]].

[0143] For example, the base station can be configured to construct each of the at least one multimodal data block by selecting the plurality of data packets out of a sequence of data packets of a first data flow and of a sequence of data packets of a second data flow based on the timing and / or context information.

[0144] FH240906PEP-2025313838. DOCX PCT 24030 17

[0145] In embodiments, the base station can be configured to execute an XR application which provides timing / context information about the multi-modal data to the lower layers. The base station then configured to construct the MMDB based on the timing / context information.

[0146] For example, the timing information can describe multi-modal events being presented to the user within the same predetermined time interval.

[0147] For example, the context information can include multi-modal events being presented to the user relating to the same presented event.

[0148] In embodiments, the base station comprises an entity configured to construct the at least one multi-modal data block.

[0149] For example, the entity can be media unit classification entity. The media unit classification entity can reside in the application layer or any lower layer, e.g., SDAP layer.

[0150] In embodiments, the reference data packet [e.g., payload data set] and the one or more secondary data packets [e.g., payload data sets] require a coordinated processing at a receiver side [e.g., in order to maintain synchronization and / or QoS coordination].

[0151] In embodiments, the base station is configured to transmit the at least one multi-modal data block.

[0152] In embodiments, the base station is configured to receive and transmit at least one signaling information, the signaling information signaling the at least one multi-modal data block.

[0153] In embodiments, the signaling information is associated with the multi-modal data block.

[0154] In embodiments, the signaling information comprises [e.g., for each data packet [e.g., payload data set]] at least one out of a multi-modal service identification, MMSID, a multi-modal data block identification, MMDB-ID.

[0155] In embodiments, the signaling information comprises for each data packet [e.g., payload data set] at least one out of a data packet type indicator [e.g., reference or secondary payload data],

[0156] FH240906PEP-2025313838. DOCX PCT 24030 18 a data flow dependency indicator [e.g., important [e.g., l-frame] or not-important [e.g., P-frame],

[0157] In embodiments, the plurality of data packets are one out of a plurality of payload data sets, a plurality of radio link control packets.

[0158] In embodiments, the base station comprises an entity [e.g., UPF] configured to generate the signaling information.

[0159] For example, the signaling I MMDB can be constructed in layer 2 of the base station (e.g., MMDB in user plane, e.g., SDAP, PDCP).

[0160] In embodiments, the base station is configured, in case that a discarding criterion of the reference data packet [e.g., payload data set] of a respective multi-modal data block is fulfilled, to discard at least one [e.g. all] of the other data packets [e.g., payload data sets] of the respective multi-modal data block.

[0161] For example, in case that a discarding criterion of the reference data packet [e.g., payload data set] of a multi-modal data block is fulfilled: (1) one or more other data packets of the respective multi-modal data block can be discarded, or (2) all other data packets belonging to one or more specific other data flows can be discarded, or (3) all other data packets of the respective multimodal data block can be cancelled.

[0162] For example, a reference flow / dataset [e.g., video] may be discardable, but a secondary flow / dataset [e.g., audio] of the multi-model flow can be maintained. In some embodiments, an additional configuration / signaling of the discarding mechanism can be used, e.g., indicating which of the flows to discard in case the discarding criterion is fulfilled.

[0163] In embodiments, the base station is configured to discard the signaling information associated with the multi-modal data block.

[0164] In embodiments, in case of a reception of an information [e.g., report or signaling or message] signaling [e.g., confirming] a successful delivery of at least the reference data packet [e.g., payload data set], the user equipment is configured to discard the at least one data packet indicated in the signaled information.

[0165] FH240906PEP-2025313838. DOCX PCT 24030 19

[0166] In embodiments, the user equipment is configured to discard all the data packets of the multimodal data block in case of the expiration of a discarding timer [e.g., associated with the reference data packet or with the entire multimodal data block],

[0167] [In embodiments, the discarding criterion is fulfilled in case of a reception of an information [e.g., report or signaling or message] signaling [e.g., confirming] a successful delivery or reception of the respective multimodal data block or signaling [e.g., confirming] a successful delivery or reception of the reference data packet [e.g., payload data set].

[0168] In embodiments, the discarding criterion is fulfilled in case of an expiration of a discarding timer associated with the multi-modal data block and / or the reference data packet.

[0169] In embodiments, the discard information (or signaled information) can be a bitmap with one bit per data packet, e.g. PDU or PDU Set.

[0170] In embodiments, in case that a data packet belonging to the respective multi-modal data block is received [e.g., received from a higher layer [e.g., higher than a layer on which the discarding is performed [e.g., PDCP layer]]] after at least one [e.g., all] of the other data packets of the respective multi modal data block was discarded, said data packet is also discarded, wherein, in case that a data packet belonging to the respective multi-modal data block was already processed [e.g., and transmitted to a lower layer [e.g., lower than a layer on which the discarding is performed [e.g., PDCP layer]]] before at least one [e.g., all] of the other data packets of the respective multi modal data block was discarded, said data packet is also discarded [e.g., in response to a respective signaling from where the discarding is performed to lower layers [e.g., from PDCP to lower layers]].

[0171] In embodiments, the signaling information comprises a discarding indicator indicating whether the discarding of at least one [e.g., all] of the data packets of the respective multi-modal data block should be performed in case that the discarding criterion is fulfilled, wherein the base station is configured to discard at least one [e.g., all] of the data packets [e.g., payload data sets] of the respective multi-modal data block only in case that a discarding indicator indicates that the discarding of the at least one [e.g., all] of the data packets of the respective multimodal data block should be performed in case that the discarding criterion is fulfilled.

[0172] In embodiments, the discarding indicator is provided as part of a QoS profile.

[0173] FH240906PEP-2025313838. DOCX PCT 24030 20

[0174] In embodiments, the base station is configured to receive the discarding indicator from a core network entity.

[0175] In embodiments, in case that a data packet of a first multi-modal data block is discarded and a data packet of a second multi-modal data block [e.g., of the same data flow] is related [or depends from] the data packet of the first multi-modal data block, then also the data packet of the second multi-modal data block is discarded.

[0176] In embodiments, the signaling information comprises a data flow dependency indicator indicating whether the data packet of the second multi-modal data block is related to the data packet of the first multi-modal data block, wherein the base station is configured to discard the data packet of the second multi-modal data block only in case that the data flow dependency indicator indicates that the data packet of the second multi-modal data block is related to the data packet of the first multi-modal data block.

[0177] In embodiments, the base station is configured to discard all other data packets of the second multi-modal data block in case that a discarding indicator indicates that the discarding of all other data packets of the second multi-modal data block should be performed in case that the data packet is discarded.

[0178] Further embodiments provide a core network entity [e.g., UPF or PCF] for processing [e.g., and / or transmitting] multi-modal data, the multi modal data including at least two data flows, wherein the multi-modal data comprises [e.g., or the core network entity is configured to construct] at least one multi-modal data block [or multi-modal data structure], each of the at least one multi-modal data blocks comprising a plurality of data packets [e.g., payload data sets] across the at least two data flows.

[0179] In embodiments, the core network entity is configured to generate and / or transmit at least one signaling information, the signaling information signaling the at least one multi-modal data block.

[0180] In embodiments, the signaling information is associated with the multi-modal data block.

[0181] In embodiments, the signaling information comprises [e.g., for each data packet [e.g., payload data set]] at least one out of a multi-modal service identification, MMSID,

[0182] FH240906PEP-2025313838. DOCX PCT 24030 21 a multi-modal data block identification, MMDB-ID.

[0183] For example, the signaling information can be shared generated by the core network and shared with the base station.

[0184] In embodiments, the signaling information comprises for each data packet [e.g., payload data set] at least one out of a data packet type indicator [e.g., reference or secondary payload data], a data flow dependency indicator [e.g., important [e.g., l-frame] or not-important [e.g., P-frame],

[0185] In embodiments, the signaling information comprises a discarding indicator indicating whether the discarding of at least one [e.g., all] of the data packets of the respective multi-modal data block should be performed in case that the discarding criterion is fulfilled.

[0186] In embodiments, the discarding indicator is provided as part of a QoS profile.

[0187] For example, the QoS profile is shared by the core network with the UE and the base station. Hence, the UE and the base station are configured to receive the QoS profile (with the discard signaling) from the core network.

[0188] Further embodiments provide a method for processing multi-modal data at a user equipment, the method comprises a step of constructing at least one multi-modal data block, each of the at least one multi-modal data blocks comprising a plurality of data packets [e.g., payload data sets] of [e.g., each of] the at least two data flows, the plurality of data packets [e.g., payload data sets] comprising a reference data packet [e.g., payload data set [e.g., reference PDU set]] of a first data flow of the at least two data flows and one or more secondary data packets [e.g., payload data sets] of at least a second data flow of the at least two data flows, wherein the secondary data packets [e.g., payload data sets] are related to [e.g., or dependent form] the reference data packet [e.g., payload data set].

[0189] Further embodiments provide a method for processing multi-modal data at a base station, the method comprises a step of constructing at least one multi-modal data block, each of the at least one multi-modal data blocks comprising a plurality of data packets [e.g., payload data sets] of [e.g., each of] the at least two data flows, the plurality of data packets [e.g., payload data sets] comprising a reference data packet [e.g., payload data set [e.g., reference PDU set]] of a first data flow of the at least two data flows and one or more secondary data packets [e.g.,

[0190] FH240906PEP-2025313838. DOCX PCT 24030 22 payload data sets] of at least a second data flow of the at least two data flows, wherein the secondary data packets [e.g., payload data sets] are related to [e.g., or dependent form] the reference data packet [e.g., payload data set].

[0191] Further embodiments provide a method for processing multi-modal data at a core network entity, the method comprises a step of processing at least one multi-modal data block, each of the at least one multi-modal data blocks comprising a plurality of data packets [e.g., payload data sets] of [e.g., each of] the at least two data flows, the plurality of data packets [e.g., payload data sets] comprising a reference data packet [e.g., payload data set [e.g., reference PDU set]] of a first data flow of the at least two data flows and one or more secondary data packets [e.g., payload data sets] of at least a second data flow of the at least two data flows, wherein the secondary data packets [e.g., payload data sets] are related to [e.g., or dependent form] the reference data packet [e.g., payload data set].

[0192] Embodiments provide an user equipment, wherein the user equipment is configured to process [e.g., and / or transmit] multi-modal data, the multi-modal data comprising multi-modal data blocks, each of the multi-modal data blocks comprising a plurality of data packets [e.g., payload data sets [e.g., PDU sets]] of at least two different data flows of the multi-modal data, the plurality of data packets [e.g., payload data sets] comprising a reference data packet [e.g. reference payload data set [e.g., reference PDU set]] and one or more secondary data packets [e.g., secondary payload data sets] that are related to [e.g., or dependent form] the reference data packet [e.g., reference payload data set], wherein the user equipment is configured, in case that a discarding criterion of the reference payload data set is fulfilled, to discard at least [e.g., all] data packets [e.g., payload data sets] of the respective multi-modal data block.

[0193] Embodiments provide a base station, wherein the base station is configured to process [e.g., and / or transmit] multi-modal data, the multi-modal data comprising multi-modal data blocks, each of the multi-modal data blocks comprising a plurality of data packets [e.g., payload data sets [e.g., PDU sets]] of at least two different data flows of the multi-modal data, the plurality of data packets [e.g., payload data sets] comprising a reference data packet [e.g. reference payload data set [e.g., reference PDU set]] and one or more secondary data packets [e.g., secondary payload data sets] that are related to [e.g., or dependent form] the reference data packet [e.g., reference payload data set], wherein the user equipment is configured, in case that a discarding criterion of the reference payload data set is fulfilled, to discard at least [e.g., all] data packets [e.g., payload data sets] of the respective multi-modal data block.

[0194] FH240906PEP-2025313838. DOCX PCT 24030 23

[0195] Embodiments described herein provide an advantage in the RAN of efficiently and effectively handling XR applications including multiple data flows with inter-dependencies. Efficiency enhancements are visible in terms of capacity or power consumption.

[0196] For that purpose, embodiments provide the concept of multi-modal data block to indicate that data packets belonging to different flows are inter-related and have thus to be processed in a coordinated manner (cf. section 2.1).

[0197] Further embodiments show how the multi-modal block can be used to enhance the legacy PDU Set discard operation with multi-modality awareness (cf. section 2.2).

[0198] 2.1 Multi-modal data block (PDU Set level)

[0199] In embodiments, the MMSID can be used to uniquely identify the XR application service. The MMSID is currently available only in the core network (CN). To enable multi-modality awareness in the RAN, in accordance with embodiments, the MMSID can be provided in the QoS profile of each QoS flow belonging to the same multi-modal service.

[0200] A QoS flow can be controlled by the Session Management Function (SMF) and can be characterized by [2]:

[0201] • A QoS profile which is provided to the RAN via the Access and Mobility Management Function (AMF) over the N2 reference point or preconfigured in the RAN.

[0202] • One or more QoS rule(s) and optionally QoS flow level QoS parameters associated with these QoS rule(s) which can be provided to the UE via the AMF over the N1 reference point and / or derived by the UE by applying reflective QoS control,

[0203] • One or more uplink and downlink packet detection rule(s) (PDR) provided by the SMF to the user plane function (UPF).

[0204] Embodiments provide the concept of multi-modal data block (MMDB). In embodiments, the MMDB indicates the data packets (e.g., PDU Sets) in the same or different QoS flows belonging to the same multi-modal service, such as, for example:

[0205] • Which are related to a reference data packet (e.g., PDU Set). The relationship between data packets (e.g., PDU Sets) belonging to different flows can be based on, e.g., timing, because the data packets (e.g., PDU Sets) are generated at around the same time by the XR application service,

[0206] FH240906PEP-2025313838. DOCX PCT 24030 24 even if a data packet (e.g., PDU Set) may a priori be associated with the data packet (e.g., PDU Set) of another flow from timing perspective, it may in fact be associated with the reference data packet (e.g., PDU Set) of the second flow because they are both referring to the same context.

[0207] • And between which synchronization and QoS coordination needs to be maintained.

[0208] Note that, in embodiments, the MMDB can enable multi-modality awareness at data packet (e.g., PDU Set) level in the RAN, as illustrated in Fig. 5.

[0209] Fig. 5 shows a schematic representation of multi-modal data 402 comprising a plurality of data flows 404-408, each data flow 404-408 comprising a plurality of data packets (e.g., payload data sets (e.g., PDU sets)), as well as an allocation of data packets of different data flows to two different multi-modal data blocks 440 and 442. As illustrated in Fig. 5 by way of example, a first data flow 404 can comprise first data packets 410_1-410_3, where the second data flow 406 can comprise second data packets 412_1-412_5, where the third data flow 408 can comprise third data packets 414_1-414_20. Note that in Fig. 5, only a portion / extract of the data flows 404-408 may be shown.

[0210] Thereby, the first multi-modal data block 403_1 can include data packet 410_1 of the first data flow 402, data packets 412_1-412_3 of the second data flow 404, and data packets 414_1 - 414_8 of the third data flow 408, where the second multi-modal data block 403_2 can include data packet 410_2 of the first data flow 402, data packets 412_4-412_5 of the second data flow 404, and data packets 414_9-414_19 of the third data flow 408.

[0211] In other words, Fig. 5 shows a multi-modal data block. Each data block corresponds to a PDU Set. One MMDB may contain multiple PDU Sets from a same QoS flow.

[0212] Note that, in embodiments, there can be exactly one reference data packet (e.g., PDU Set) per MMDB. For each MMDB, the reference data packet (e.g., PDU Set) belongs to the same flow (i.e., reference flow). For example, referring to Fig. 5, data packet 410_1 of the first data flow 402 can be the reference data packet of the first multi-modal data block 403_1 , where data packet 410_2 of the first data flow 402 can be the reference data packet of the second multi-modal data block 403_2.

[0213] In some contexts, one mode (or flow) may be more important than the others. Typically, for some AA / applications, the user experience may be more impacted if the audio stream has

[0214] FH240906PEP-2025313838. DOCX PCT 24030 25 issues than if the video stream has issues, such as, for example, a YouTube video with perfect video but noise or dropouts in the audio.

[0215] Note that, in embodiments, all the (other) data packets (e.g., PDU Sets) within the MMDB may depend on the reference data packet (e.g., PDU Set). However, there may not necessarily be any dependency between them.

[0216] Further, note that, in embodiments, without loss of generality, the MMDB can also include more than one data packet (e.g., PDU Set) belonging to the same flow as the reference data packet (e.g., PDU Set). An example is shown in Fig. 6.

[0217] Fig. 6 shows a schematic representation of multi-modal data 402 comprising a plurality of data flows 404-408, each data flow 404-408 comprising a plurality of data packets (e.g., payload data sets (e.g., PDU sets)), as well as an allocation of data packets of different data flows to a multi-modal data block. Compared to Fig. 5, in the embodiment of Fig. 6, by way of example, only one multi-modal data block 403 is shown, where the multi-modal data block 403 comprises data packets 410_1-410_2 of the first data flow 404, data packets 412_1-412_5 of the second data flow, and data packets 414_1-414_19 of the third data flow. Note that in Fig. 6, only a portion / extract of the data flows 404-408 may be shown.

[0218] In other words, Fig. 6 shows an alternative example of the multi-modal data block with more than one data packet (e.g., PDU Set) belonging to the same flow as the reference data packet (e.g., PDU Set).

[0219] In embodiments, for multi-modal services, the MMDB-related parameters can be provided for each data packet (e.g., PDU Set). Thus, embodiments add at least some of the information provided in Table 1 to the data packet (e.g., PDU Set) information, to enable multi-modality awareness in the RAN.

[0220] FH240906PEP-2025313838. DOCX PCT 24030 26

[0221] Table 1 - Example of MMDB-related parameters added to the PDU Set information to support multi-modality awareness

[0222] In embodiments, it is up to the XR application service to provide the information regarding the association between the data packets (e.g., PDU Sets) of the different flows belonging to the same XR application service (i.e. , the parameters mmdb_ref_flow and flow_dependence).

[0223] Note that, in the downlink, the data packet (e.g., PDU Set) information may be sent by the UPF to the RAN in the GTP-U header [5], The data packet (e.g., PDU Set) information can include the new multi-modal information provided in Table 1. It is the role of the SMF to instruct the UPF to perform the MMDB marking. Moreover, the RAN may indicate its support for MMDB handling.

[0224] Further, note that, in the uplink, the UE may need to be able to identify data packet (e.g., PDU Sets) dynamically, including PSI, and the inter-dependencies between data packet (e.g., PDU Sets) belonging to different flows, and perform the MMDB marking accordingly, include the new multi-modal information provided in Table 1.

[0225] The advantage of identifying multi-modality on the RAN is that this inter-dependency information on XR application service provides a finer granularity available on lower layers, e.g., between the data packets (e.g., PDU Sets) within and / or between the flows. This way, the synchronization between the different multi-modalities can be improved or even optimized, e.g., related to latency, resulting in an improved user experience.

[0226] In embodiments, the MMDB concept may apply not only to PDU Sets but also to any type of packets (e.g., RLC), without loss of generality.

[0227] 2.2 Data packet (e.g., PDU Set) discard operation

[0228] FH240906PEP-2025313838. DOCX PCT 24030 27

[0229] In section 2.1 , the concept of MMDB is described as a means to enable multi-modality awareness in the RAN. Embodiments can use the MMDB to enhance the data packet (e.g., PDU set) discard operation. More specifically, since the reference data packet (e.g., PDU Set) has dependencies with one or multiple other data packet(s) (e.g., PDU Set(s)) within the same single-mode flow or across the multi-modal flows of the XR application service, the two following cases can be considered if this data packet (e.g., PDU Set) is discarded:

[0230] • If mmdb_discard is configured: o If the discardTimer of the reference PDU Set of the MMDB (or of the entire MMDB) expires, all PDU Sets of the MMDB can be discarded, or o if the receiving PDCP entity confirms the successful delivery of at least the reference PDU Set, the transmitting PDCP entity can discard the PDU Sets indicated in the PDCP status report.

[0231] Moreover, all remaining PDU sets within the MMDB can also be discarded to avoid unnecessary data transmission and to save energy, in particular: o PDU Sets subsequently received from upper layers can also be discarded if they belong to the MMDB, and / or o if the corresponding MMDB has already been submitted to lower layers, the discard can be indicated to lower layers.

[0232] • Otherwise, the legacy procedure may apply, i.e. , no other data packet (e.g., PDU Set) within the MMDB is discarded (excepted if they meet the legacy data packet (e.g., PDU Set) discard criteria). Only if flow_dependence is configured, the data packets (e.g., PDU Sets) belonging to the same flow are discarded.

[0233] The following Table 2 provides an example of a parameter for the MMDB discard operation.

[0234] Table 2

[0235] FH240906PEP-2025313838. DOCX PCT 24030 28

[0236] In embodiments, the mmdb_discard indicator can be provided as part of the QoS profile provided to by the PCF to the AMF (via the SMF). It may be also included in the QoS rules and optionally QoS flow level QoS parameters associated with these QoS rule(s) which are provided to the UE.

[0237] Subsequently, an example of a data packet (e.g., PDU set) discard mechanism is described. Thereby, by way of example, it is considered the case when a dependency exists between two data packets (e.g., PDU Sets) of the same QoS flow. Moreover, the two data packets (e.g., PDU Sets) belong to two different MMDBs.

[0238] Note that this is the case, for example, between a P-frame and the l-frame (or a previous P- frame) from the same GOP (see section 1.1).

[0239] The following example shows how the data packet (e.g., PDU Set) discard operation works within and across MMDBs. Thereby, it is made reference to the embodiment of the two multimodal data blocks 403_1 and 403_2 of Fig. 7 and it is assumed by way of example that there is a dependency from data packet 410_2 of the second multi-modal data block 403_2 to data packet 410_1 of the first multi-modal data block 403_1 .

[0240] As indicated in Fig. 7, as a first step, it can be assumed that the reference data packet (e.g., PDU Set) 410_1 of the first data flow 404 (flow #1) is either lost or associated to a discarded SDU. This data packet (e.g., PDU Set) belongs to the first multi-modal data block 403_1 (MMDB #1).

[0241] As indicated in Fig. 8, as a second step, the multi-modal data packet (e.g., PDU Set) discard operation can be performed within the first multi-modal data block 403_1 (MMDB #1). For first multi-modal data block 403_1 (MMDB #1), the parameter mmdb_discard is configured, i.e., all the data packets (e.g., PDU Sets) in the first multi-modal data block 403_1 (MMDB #1) are discarded.

[0242] As indicated in Fig. 9, as a third step, the data packet (e.g., PDU Set) 410_2 of the first data flow 404 (flow #1) in the second multi-modal data block 403_2 (MMDB #2) is dependent on the data packet (e.g., PDU Set) 410_1 of the first data flow 408 (flow #1) in the first multi-modal data block 403_1 (MMDB #1), i.e., it cannot be decoded independently. Since the latter has been previously discarded during the second step, the data packet (e.g., PDU Set) 410_2 of the first data flow 404 (flow #1) in the second multi-modal data block 403_2 (MMDB #2) is now useless and it should also be discarded.

[0243] FH240906PEP-2025313838. DOCX PCT 24030 29

[0244] As indicated in Fig. 10, as a fourth step, the multi-modal data packet (e.g., PDU Set) discard operation is now performed within the second multi-modal data block 403_2 (MMDB #2). Thereby, in Fig. 10 it es exemplarily assumed that for the second multi-modal data block 403_2 (MMDB #2), the parameter mmdb_discard is not configured, i.e. , no other data packets (e.g., PDU Sets) in the second multi-modal data block 403_2 (MMDB #2) are discarded.

[0245] Note that, in embodiments, the data packet (e.g., PDU Set) discard operation can be performed at the transmitting PDCP entity. However, in embodiments, it can also be used at the receiving PDCP entity. When the reference data packet (e.g., PDU Set) of a MMDB is received too late or not decoded successfully, the receiving PDCP entity may not waste time and resources for the reception of the other data packets (e.g., PDU Sets) of the MMDB, i.e.,

[0246] • If mmdb_discard is configured:

[0247] • If the discardTimer of the reference PDU Set of the MMDB (or of the entire MMDB) expires, all PDU Sets of the MMDB can be discarded, or

[0248] • if the receiving PDCP entity confirms the successful delivery of at least the reference PDU Set, the transmitting PDCP entity can discard the PDU Sets indicated in the PDCP status report.

[0249] Moreover, all remaining PDU sets within the MMDB can also be discarded to avoid unnecessary data transmission and to save energy, in particular:

[0250] • PDU Sets subsequently received from upper layers can also be discarded if they belong to the MMDB, and / or

[0251] • if the corresponding MMDB has already been submitted to lower layers, the discard can be indicated to lower layers.

[0252] • otherwise, only the data packets (e.g., PDU Sets) belonging to the same flow can be discarded if flow_dependence is configured. Data packets (e.g., PDU Sets) belonging to other flows are not discarded.

[0253] 2.3 Implementation in Uplink and Downlink

[0254] For uplink and downlink transmissions, the flow of the multi-modality information in accordance with embodiments between peer XR application entities can be summarized as illustrated in Figs. 16 and 17, respectively.

[0255] Specifically, Fig. 11 shows a schematic representation of an uplink QoS model with an extension for processing / transmitting multi-modal data. Thereby, Fig. 11 is adapted version of Fig. 5.7.1.5-1 of [2], As shown in Fig. 11 , the uplink QoS model can include the following

[0256] FH240906PEP-2025313838. DOCX PCT 24030 30 entities: a first XR application / service 502, a user equipment 504, a radio access network 506 (RAN), an access and mobility function 508 (AMF), a session management function 510 (SMF), a policy control function 512 (PCF), an user plane function 514 (UPF), and a second XR application / service 516.

[0257] As shown in Fig. 11 , the first XR application / service 502 can provide multi-modal data (data packets from application) to the user equipment 504, where the user equipment 504 can be configured to process the multi-modal data using one or more of the following entities: a media classification entity 520, a data packet (e.g., PDU set) classification entity 522, a policy enforcement entity 524 and QoS rules 526.

[0258] For example, the user equipment 504 can be configured to construct the at least one multimodal data block using the data packet (e.g., PDU set) classification entity 522, or in other words, to associate the data packets (e.g., PDU sets) as described above in section 2.1.

[0259] For example, the user equipment 504 can be configured to generate the signaling information (e.g., including one or more of the multi-modal parameters: mmsid, mmdb-id, mmdb_ref_flow, flow dependence) as described above in section 2.1 using the data packet (e.g., PDU set) classification entity 522.

[0260] For example, the user equipment can be configured to perform the data packet (e.g., PDU set) discarding operation as described above in section 2.2 after the policy enforcement entity 524.

[0261] Further, as shown in Fig. 11 , the second XR application / service 516 can be configured to provide control information (e.g., MMDB control information) allowing to perform the data packet (e.g., PDU set) discarding operation as described above in section 2.2.

[0262] Fig. 12 shows a schematic representation of a downlink QoS model with an extension for processing / transmitting multi-modal data. Thereby, Fig. 12 is adapted version of Fig. 5.7.1.5- 1 of [2], As shown in Fig. 12, the downlink QoS model can include the following entities: a first XR application / service 502, a user equipment 504, a radio access network 506 (RAN), an access and mobility function 508 (AMF), a session management function 510 (SMF), a policy control function 512 (PCF), a user plane function 514 (UPF), and a second XR application / service 516.

[0263] As shown in Fig. 12, the second XR application / service 516 can provide multi-modal data (data packets from application) to the user plane function 514 (UPF), where the user plane function

[0264] FH240906PEP-2025313838. DOCX PCT 24030 31

[0265] 514 (UPF) can be configured to process the multi-modal data using one or more of the following entities: a media packet filter 530, a media packet classifier 532, a PDU set identification entity 534, and a packet detection rules entity 536.

[0266] For example, the user plane function 514 (UPF) can be configured to generate the signaling information (e.g., including one or more of the multi-modal parameters: mmsid, mmdb-id, mmdb_ref_flow, flow dependence) as described above in section 2.1 using the packet detection rules entity 536.

[0267] For example, the radio access network 506 (RAN) can be configured to perform the MMDB marking as described above in section 2.1.

[0268] For example, the second XR application / service 516 can be configured to provide control information (e.g., MMDB control information) allowing to perform the data packet (e.g., PDU set) discarding operation as described above in section 2.2.

[0269] 3. Definitions

[0270] Subsequently, definitions and examples are provided which might apply in some of the embodiments described herein.

[0271] In embodiments, 3DoF describes a three rotational and un-limited movement around the X, Y and Z axes (respectively pitch, yaw and roll).

[0272] In embodiments, 6DoF describes 3DoF with full translational movements along X, Y and Z axes. Beyond the 3DoF experience, it adds (i) moving up and down (elevating / heaving); (ii) moving left and right (strafing / swaying); and (iii) moving forward and backward (walking / surging).

[0273] In embodiments, a group of pictures (GOP) can be a collection of consecutive video frames.

[0274] In embodiments, a synchronization threshold can describe a maximum tolerable temporal separation of the onset of two stimuli, one of which is presented to one sense and the other to another sense, such that the accompanying sensory objects are perceived as being synchronous. Typical synchronization thresholds for immersive multi-modality VR applications are provided in the below table [1]:

[0275] FH240906PEP-2025313838. DOCX PCT 24030 32

[0276] In embodiments, a PDU Set can comprise one or more PDlls carrying the payload of one unit of information, for example, generated at the application level (e.g., frame(s) or a video slice(s), etc., for XR Services). All the PDlls of a PDU set can be transmitted within the same QoS Flow.

[0277] In embodiments, a PDU Set Information can be sent, for example, by the PSA UPF to the RAN in the GTP-U header. It is used by the RAN for PDU Set based QoS handling and comprises, for example, one or more out of:

[0278] PDU Set Sequence Number,

[0279] Indication of End PDU of the PDU Set,

[0280] PDU Sequence Number within a PDU Set,

[0281] PDU Set Size in bytes,

[0282] PDU Set Importance, which identifies the relative importance of a PDU Set compared to other PDU Sets within a QoS Flow.

[0283] In embodiments, PDU Set quality-of-service (QoS) parameters can be determined, for example, by the PCF based on information provided by AF (and / or local configuration). They can be sent to the SMF as part of the PCC rule and, in turns, provided to the RAN as part of the QoS profile. It includes, for example, one or more out of:

[0284] PDU Set delay budget (PSDB),

[0285] PDU Set error rate (PSER),

[0286] PDU Set integrated handling information (PSI HI).

[0287] In embodiments, a PDU Set delay budget (PSDB) can be the time between reception of the first PDU (at the UPF in downlink, at the UE in uplink) and the successful delivery of the last arrived PDU of a PDU Set (at the UE in downlink, at the UPF in uplink).

[0288] In embodiments, a PDU Set integrated handling indication (PSIHI) can indicate whether all PDUs of the PDU Set are needed for the usage of PDU Set by application layer.

[0289] FH240906PEP-2025313838. DOCX PCT 24030 33

[0290] In embodiments, a data burst can be a set of multiple PDlls generated and sent by the application in a short period of time. A data burst can be composed of one or multiple PDU Sets.

[0291] In embodiments, a multi-modal service can be a communication service that comprises of several data flows that relate to each other and that are subject to application coordination. The data flows can transfer different types of data (for example audio, video, positioning, haptic data) and may come from different sources (e.g., a single UE, a single device or multiple devices connected to the single UE, or multiple UEs).

[0292] For the single UE case, it is expected that those data flows are closely related and require strong application coordination for the proper execution of the multi-modal application and therefore, all those data flows are transmitted in a single PDU session

[0293] In embodiments, a multi-modal service ID (MMSID) can be an explicit indication that data flows are related to a multi-modal service [2],

[0294] In embodiments, the PCF may use this information to derive the correct PCC rules and to apply appropriate QoS policies for the data flows that are part of a specific multi-modal application.

[0295] In embodiments, the 5G QoS characteristics describe the packet forwarding treatment that a QoS Flow receives edge-to-edge between the UE and the UPF in terms of one or more out of the following performance characteristics:

[0296] Resource type (Non-GBR, GBR, Delay-critical GBR),

[0297] Priority level,

[0298] Packet delay budget (including Core Network packet delay budget),

[0299] Packet error rate,

[0300] Averaging window (for GBR and delay-critical GBR resource type only), Maximum data burst volume (for delay-critical GBR resource type only).

[0301] In embodiments, the QoS flow can be the finest granularity of QoS differentiation in the PDU Session. A QoS flow ID (QFI) can be used to identify a QoS flow in the 5G System. User Plane traffic with the same QFI within a PDU Session receives the same traffic forwarding treatment (e.g., scheduling, admission threshold) [2], Any QoS flow can be characterized, for example, by one or more out of:

[0302] FH240906PEP-2025313838. DOCX PCT 24030 34

[0303] A QoS profile provided by the SMF to the RAN via the AMF over the N2 reference point or preconfigured in the RAN.

[0304] One or more QoS rule(s) and optionally QoS flow level QoS parameters associated with these QoS rule(s) which can be provided by the SMF to the UE via the AMF over the N1 reference point and / or derived by the UE by applying Reflective QoS control. One or more uplink and downlink PDR(s) provided by the SMF to the UPF.

[0305] In embodiments, within the 5GS, a QoS flow associated with the default QoS rule can be required to be established for a PDU Session and remains established throughout the lifetime of the PDU Session. A QoS flow is associated with QoS requirements as specified by QoS parameters and QoS characteristics.

[0306] 4. Further embodiments

[0307] Various elements and features of the present invention may be implemented in hardware using analog and / or digital circuits, in software, through the execution of instructions by one or more general purpose or special-purpose processors, or as a combination of hardware and software. For example, embodiments of the present invention may be implemented in the environment of a computer system or another processing system. Fig. 13 illustrates an example of a computer system 500. The units or modules as well as the steps of the methods performed by these units may execute on one or more computer systems 500. The computer system 500 includes one or more processors 502, like a special purpose or a general-purpose digital signal processor. The processor 502 is connected to a communication infrastructure 504, like a bus or a network. The computer system 500 includes a main memory 506, e.g., a random-access memory (RAM), and a secondary memory 508, e.g., a hard disk drive and / or a removable storage drive. The secondary memory 508 may allow computer programs or other instructions to be loaded into the computer system 500. The computer system 500 may further include a communications interface 510 to allow software and data to be transferred between computer system 500 and external devices. The communication may be in the from electronic, electromagnetic, optical, or other signals capable of being handled by a communications interface. The communication may use a wire or a cable, fiber optics, a phone line, a cellular phone link, an RF link and other communications channels 512.

[0308] The terms “computer program medium” and “computer readable medium” are used to generally refer to tangible storage media such as removable storage units or a hard disk installed in a hard disk drive. These computer program products are means for providing software to the computer system 500. The computer programs, also referred to as computer

[0309] FH240906PEP-2025313838. DOCX PCT 24030 35 control logic, are stored in main memory 506 and / or secondary memory 508. Computer programs may also be received via the communications interface 510. The computer program, when executed, enables the computer system 500 to implement the present invention. In particular, the computer program, when executed, enables processor 502 to implement the processes of the present invention, such as any of the methods described herein. Accordingly, such a computer program may represent a controller of the computer system 500. Where the disclosure is implemented using software, the software may be stored in a computer program product and loaded into computer system 500 using a removable storage drive, an interface, like communications interface 510.

[0310] The implementation in hardware or in software may be performed using a digital storage medium, for example cloud storage, a floppy disk, a DVD, a Blue-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.

[0311] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0312] Generally, embodiments of the present invention may be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine-readable carrier.

[0313] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine-readable carrier. In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0314] A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data

[0315] FH240906PEP-2025313838. DOCX PCT 24030 36 communication connection, for example via the Internet. A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein. A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0316] In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

[0317] The above described embodiments are merely illustrative for the principles of the present invention. It is understood that modifications and variations of the arrangements and the details described herein are apparent to others skilled in the art. It is the intent, therefore, to be limited only by the scope of the impending patent claims and not by the specific details presented by way of description and explanation of the embodiments herein.

[0318] FH240906PEP-2025313838. DOCX PCT 24030 37

[0319] List of References

[0320] [1] 3GPP TS 22 261 v19.7.0, “Service requirements for the 5G system,” Jun. 2024.

[0321] [2] 3GPP TS 23 501 v19.0.0, “5G; System architecture for the 5G System (5GS),” Jun. 2024.

[0322] [3] 3GPP TR 38 838 v17.0.0, “Study on XR (Extended Reality) evaluations for NR,” Dec. 2021.

[0323] [4] 3GPP TR 38323 v18.2.0, “Packet Data Convergence Protocol (PDCP) specification,” Jun. 2024.

[0324] [5] 3GPP TR 38 835 v18.0.1 , “Study on XR enhancements for NR,” Apr. 2023.

[0325] [6] 3GPP TR 26.928 v18.0.0, “Extended Reality (XR) in 5G,” Mar. 2023.

[0326] [7] RFC 3550, “RTP; A Transport Protocol for Real-Time Applications.”

[0327] FH240906PEP-2025313838. DOCX PCT 24030 38

[0328] Abbreviations

[0329] 3DoF three degrees of freedom

[0330] 3GPP third generation partnership project

[0331] 6D0F six degrees of freedom

[0332] AMF access and mobility management function

[0333] BS base station

[0334] BSD bucket size duration

[0335] CN core network

[0336] D2D device-to-device

[0337] DC dual connectivity

[0338] DCI downlink control information

[0339] DL downlink

[0340] DoF degree of freedom

[0341] DRS discovery reference signal

[0342] DRX discontinuous reception

[0343] DTX discontinuous transmission

[0344] E2E end-to-end eNB evolved node B fps frames per second

[0345] FR frequency range

[0346] FR1 frequency range one

[0347] FR2 frequency range two gNB next generation node B

[0348] GOP group of pictures

[0349] GPRS general packet radio service

[0350] GSCN global synchronization channel number

[0351] GTP GPRS tunnelling protocol

[0352] GTP-U GTP user

[0353] HARQ hybrid automatic repeat request

[0354] ID identity

[0355] IFFT inverse fast Fourier transform loT internet of things

[0356] LCH logical channel

[0357] LTE long-term evolution

[0358] MAC medium access control

[0359] FH240906PEP-2025313838. DOCX PCT 24030 39

[0360] MAC-CE medium access control - control element

[0361] MIB master information block

[0362] MMDB multi-modal data block

[0363] MMDB-ID multi-modal data block identity

[0364] MMSID multi-modal service ID

[0365] NR new radio

[0366] NTP network time protocol

[0367] OFDM orthogonal frequency-division multiplexing

[0368] OFDMA orthogonal frequency-division multiple access

[0369] PBCH physical broadcast channel

[0370] PBR prioritized bit rate

[0371] PC5 interface using the sidelink channel for D2D communication

[0372] PCC policy and charging control

[0373] PDCCH physical downlink control channel

[0374] PDCP packet data convergence protocol

[0375] PDR packet detection rule

[0376] PDSCH physical downlink shared channel

[0377] PDU packet data unit

[0378] PRACH physical random access channel

[0379] PSA PDU session anchor

[0380] PSBCH physical sidelink broadcast channel

[0381] PSCCH physical sidelink control channel

[0382] PSDB PDU set delay budget

[0383] PSI PDU set importance

[0384] PSIHI PDU set integrated handling indication

[0385] PSSCH physical sidelink shared channel

[0386] PUCCH physical uplink control channel

[0387] PUSCH physical uplink shared channel

[0388] QFI QoS flow identifier

[0389] QoE quality of experience

[0390] QoS quality of service

[0391] RACH random access channel

[0392] RAN radio access network

[0393] RE resource element

[0394] RLC radio link control

[0395] RRC radio resource control

[0396] RS reference signal

[0397] FH240906PEP-2025313838. DOCX PCT 24030 40

[0398] RSRP reference signal received power

[0399] RSRQ reference signal received quality

[0400] RSS received signal strength

[0401] RTP real-time transport protocol

[0402] SCI sidelink control information

[0403] SDU service data unit

[0404] SI system information

[0405] SIB system information block

[0406] SL sidelink

[0407] SMF session management function

[0408] SSB synchronization signal block

[0409] SSS secondary synchronization signal

[0410] S-SSB sidelink synchronization signal block sTTI short transmission time interval

[0411] TDD time division duplex

[0412] TTI transmission time interval

[0413] UE user equipment

[0414] UL uplink

[0415] UMTS universal mobile telecommunication system

[0416] UPF user plane function

[0417] V2V vehicle-to-vehicle

[0418] V2X vehicle-to-everything

[0419] VR virtual reality

[0420] XR extended reality

[0421] FH240906PEP-2025313838. DOCX PCT

Claims

24030 41Claims1. User equipment (202i), for processing multi-modal data (402), the multi modal data (402) including at least two data flows (404, 406, 408), wherein the user equipment (202i) is configured to construct at least one multi-modal data block (403_1 , 403_2), each of the at least one multi-modal data blocks (403_1 , 403_2) comprising a plurality of data packets of the at least two data flows (404, 406, 408), the plurality of data packets comprising a reference data packet of a first data flow (404) of the at least two data flows (404, 406, 408) and one or more secondary data packets of at least a second data flow (406, 408) of the at least two data flows (404, 406, 408), wherein the secondary data packets are related to the reference data packet.

2. User equipment (202i) according to claim 1 , wherein the at least two different data flows (404, 406, 408) are associated with different types of information.

3. User equipment (202i) according to claim 1 or 2, wherein each of the at least two different data flows (404, 406, 408) comprises a sequence of data packets.

4. User equipment (202i) according to claim 3, wherein the reference data packet is a proper subset of the sequence of data packets of the first data flow (404), wherein the one or more secondary data packets are a proper subset of the sequence of data packets of the second data flow (406, 408).

5. User equipment (202i) according to one of the claims 1 to 4, wherein the one or more secondary data packets are related to the reference data packet based on timing or context.

6. User equipment (202i) according to one of the claims 1 to 5,FH240906PEP-2025313838. DOCX PCT24030 42 wherein the user equipment (202i) is configured to construct the at least one multimodal data block (403_1 , 403_2) based on a timing information and / or a context information.

7. User equipment (202i) according to one of the claims 1 to 6, wherein the user equipment (202i) comprises an entity configured to construct the at least one multi-modal data block (403_1 , 403_2).

8. User equipment (202i) according to one of the claims 1 to 7, wherein the reference data packet and the one or more secondary data packets require a coordinated processing at a receiver side.

9. User equipment (202i) according to one of the claims 1 to 8, wherein the user equipment (202i) is configured to transmit the at least one multimodal data block (403_1 , 403_2).

10. User equipment (202i) according to one of the claims 1 to 9, wherein the user equipment (202i) is configured to transmit at least one signaling information, the signaling information signaling the at least one multi-modal data block (403_1, 403_2),11 . User equipment (202i) according to claim 10, wherein the signaling information comprises at least one out of a multi-modal service identification, MMSID, a multi-modal data block identification, MMDB-ID.

12. User equipment (202i) according to claim 10 or 11 , wherein the signaling information comprises for each data packet at least one out of a data packet type indicator, a data flow dependency indicator.FH240906PEP-2025313838. DOCX PCT24030 4313. User equipment (202i) according to one of the claims 10 to 12, wherein the user equipment (202i) comprises a data packet classification entity configured to generate the signaling information.

14. User equipment (202i) according to one of the claims 1 to 13, wherein the user equipment (202i) is configured, in case that a discarding criterion of the reference data packet of a respective multi-modal data block is fulfilled, to discard at least one of the other data packets of the respective multi-modal data block.

15. User equipment (202i) according to claim 14, wherein the user equipment (202i) is configured to discard the signaling information associated with the multi-modal data block.

16. User equipment (202i) according to claim 14 or 15, in case of a reception of an information signaling a successful delivery of at least the reference data packet, the user equipment (202i) is configured to discard the at least one data packet indicated in the signaled information, or wherein the user equipment (202i) is configured to discard all the data packets of the multi-modal data block in case of the expiration of a discarding timer.

17. User equipment (202i) according to one of the claims 14 to 16, wherein, in case that a data packet belonging to the respective multi-modal data block is received after at least one of the other data packets of the respective multi modal data block was discarded, said data packet is also discarded, wherein, in case that a data packet belonging to the respective multi-modal data block was already processed before at least one of the other data packets of the respective multi modal data block was discarded, said data packet is also discarded.

18. User equipment (202i) according to one of the claims 14 to 17,FH240906PEP-2025313838. DOCX PCT24030 44 wherein the signaling information comprises a discarding indicator indicating whether the discarding of at least one of the data packets of the respective multi-modal data block should be performed in case that the discarding criterion is fulfilled, wherein the user equipment (202i) is configured to discard at least one of the data packets of the respective multi-modal data block only in case that a discarding indicator indicates that the discarding of the at least one of the data packets of the respective multi-modal data block should be performed in case that the discarding criterion is fulfilled.

19. User equipment (202i) according to claim 18, wherein the discarding indicator is provided as part of a QoS profile.

20. User equipment (202i) according to claim 18 or 19, wherein the user equipment (202i) is configured to receive the discarding indicator from a core network entity.21 . User equipment (202i) according to one of the claims 14 to 20, wherein, in case that a data packet of a first multi-modal data block is discarded and a data packet of a second multi-modal data block is related the data packet of the first multi-modal data block, then also the data packet of the second multi-modal data block is discarded.

22. User equipment (202i) according to claim 21 , wherein the signaling information comprises a data flow dependency indicator indicating whether the data packet of the second multi-modal data block is related to the data packet of the first multi-modal data block, wherein the user equipment (202i) is configured to discard the data packet of the second multi-modal data block only in case that the data flow dependency indicator indicates that the data packet of the second multi-modal data block is related to the data packet of the first multi-modal data block.FH240906PEP-2025313838. DOCX PCT24030 4523. User equipment (202i) according to claim 21 or 22, wherein the user equipment (202i) is configured to discard all other data packets of the second multi-modal data block in case that a discarding indicator indicates that the discarding of all other data packets of the second multi-modal data block should be performed in case that the data packet is discarded.

24. Base station (200), for processing multi-modal data (402), the multi modal data including at least two data flows (404, 406, 408), wherein the base station (200) is configured to construct at least one multi-modal data block (403_1 , 403_2), each of the at least one multi-modal data blocks (403_1 , 403_2) comprising a plurality of data packets of the at least two data flows (404, 406, 408), the plurality of data packets comprising a reference data packet of a first data flow (404) of the at least two data flows (404, 406, 408) and one or more secondary data packets of at least a second data flow (406, 408) of the at least two data flows (404, 406, 408), wherein the secondary data packets are related to the reference data packet.

25. Base station (200) according to claim 24, wherein the at least two different data flows (404, 406, 408) are associated with different types of information.

26. Base station (200) according to claim 24 or 25, wherein each of the at least two different data flows (404, 406, 408) comprises a sequence of data packets.

27. Base station (200) according to claim 26, wherein the reference data packet is a proper subset of the sequence of data packets of the first data flow (404), wherein the one or more secondary data packets are a proper subset of the sequence of data packets of the second data flow (406, 408).FH240906PEP-2025313838. DOCX PCT24030 4628. Base station (200) according to one of the claims 24 to 27, wherein the one or more secondary data packets are related to the reference data packet based on timing or context.

29. Base station (200) according to one of the claims 24 to 28, wherein the base station (200) is configured to construct the at least one multi-modal data block (403_1 , 403_2) based on a timing and / or context information.

30. Base station (200) according to claim 29, wherein the base station (200) comprises an entity configured to construct the at least one multi-modal data block (403_1 , 403_2).31 . Base station (200) according to one of the claims 24 to 30, wherein the reference data packet and the one or more secondary data packets require a coordinated processing at a receiver side.

32. Base station (200) according to one of the claims 24 to 31 , wherein the base station (200) is configured to transmit the at least one multi-modal data block (403_1 , 403_2).

33. Base station (200) according to one of the claims 24 to 32, wherein the base station (200) is configured to receive and transmit at least one signaling information, the signaling information signaling the at least one multi-modal data block (403_1 , 403_2),34. Base station (200) according to claim 33, wherein the signaling information comprises at least one out of a multi-modal service identification, MMSID, a multi-modal data block identification, MMDB-ID.FH240906PEP-2025313838. DOCX PCT24030 4735. Base station (200) according to claim 33 or 34, wherein the signaling information comprises for each data packet at least one out of a data packet type indicator, a data flow dependency indicator.

36. Base station (200) according to one of the claims 33 to 35, wherein the base station (200) comprises an entity configured to generate the signaling information.

37. Base station (200) according to one of the claims 24 to 36, wherein the base station (200) is configured, in case that a discarding criterion of the reference data packet of a respective multi-modal data block is fulfilled, to discard at least one of the other data packets of the respective multi-modal data block.

38. Base station (200) according to claim 37, wherein the base station (200) is configured to discard the signaling information associated with the multi-modal data block.

39. Base station (200) according to claim 37 or 38, wherein in case of a reception of an information signaling a successful delivery of at least the reference data packet, the user equipment (202i) is configured to discard the at least one data packet indicated in the signaled information, or wherein the user equipment (202i) is configured to discard all the data packets of the multi-modal data block in case of the expiration of a discarding timer.

40. Base station (200) according to one of the claims 37 to 39, wherein, in case that a data packet belonging to the respective multi-modal data block is received after at least one of the other data packets of the respective multi modal data block was discarded, said data packet is also discarded,FH240906PEP-2025313838. DOCX PCT24030 48 wherein, in case that a data packet belonging to the respective multi-modal data block was already processed before at least one of the other data packets of the respective multi modal data block was discarded, said data packet is also discarded.41 . Base station (200) according to one of the claims 37 to 40, wherein the signaling information comprises a discarding indicator indicating whether the discarding of at least one of the data packets of the respective multi-modal data block should be performed in case that the discarding criterion is fulfilled, wherein the base station (200) is configured to discard at least one of the data packets of the respective multi-modal data block only in case that a discarding indicator indicates that the discarding of the at least one of the data packets of the respective multi-modal data block should be performed in case that the discarding criterion is fulfilled.

42. Base station (200) according to claim 41 , wherein the discarding indicator is provided as part of a QoS profile.

43. Base station (200) according to claim 41 or 42, wherein the base station (200) is configured to receive the discarding indicator from a core network entity.

44. Base station (200) according to one of the claims 37 to 43, wherein, in case that a data packet of a first multi-modal data block is discarded and a data packet of a second multi-modal data block is related the data packet of the first multi-modal data block, then also the data packet of the second multi-modal data block is discarded.

45. Base station (200) according to claim 44,FH240906PEP-2025313838. DOCX PCT24030 49 wherein the signaling information comprises a data flow dependency indicator indicating whether the data packet of the second multi-modal data block is related to the data packet of the first multi-modal data block, wherein the base station (200) is configured to discard the data packet of the second multi-modal data block only in case that the data flow dependency indicator indicates that the data packet of the second multi-modal data block is related to the data packet of the first multi-modal data block.

46. Base station (200) according to claim 44 or 45, wherein the base station (200) is configured to discard all other data packets of the second multi-modal data block in case that a discarding indicator indicates that the discarding of all other data packets of the second multi-modal data block should be performed in case that the data packet is discarded.

47. Core network entity for processing multi-modal data, the multi modal data including at least two data flows (404, 406, 408), wherein the multi-modal data comprises at least one multi-modal data block (403_1 , 403_2), each of the at least one multi-modal data blocks (403_1 , 403_2) comprising a plurality of data packets across the at least two data flows (404, 406, 408).

48. Core network entity according to claim 47, wherein the core network entity is configured to generate and / or transmit at least one signaling information, the signaling information signaling the at least one multi-modal data block (403_1 , 403_2),49. Core network entity according to claim 48, wherein the signaling information comprises at least one out of a multi-modal service identification, MMSID, a multi-modal data block identification, MMDB-ID.

50. Core network entity according to claim 48 or 49,FH240906PEP-2025313838. DOCX PCT24030 50 wherein the signaling information comprises for each data packet at least one out of a data packet type indicator, a data flow dependency indicator.51 . Core network entity according to claim 47, wherein the signaling information comprises a discarding indicator indicating whether the discarding of at least one of the data packets of the respective multi-modal data block should be performed in case that the discarding criterion is fulfilled.

52. Core network entity according to claim 51 , wherein the discarding indicator is provided as part of a QoS profile.

53. Method for processing multi-modal data at a user equipment (202i), the method comprising: constructing at least one multi-modal data block (403_1 , 403_2), each of the at least one multi-modal data blocks (403_1 , 403_2) comprising a plurality of data packets of the at least two data flows (404, 406, 408), the plurality of data packets comprising a reference data packet of a first data flow (404) of the at least two data flows (404, 406, 408) and one or more secondary data packets of at least a second data flow (406, 408) of the at least two data flows (404, 406, 408), wherein the secondary data packets are related to the reference data packet.

54. Method for processing multi-modal data at a base station (200), the method comprising: constructing at least one multi-modal data block (403_1 , 403_2), each of the at least one multi-modal data blocks (403_1 , 403_2) comprising a plurality of data packets of the at least two data flows (404, 406, 408), the plurality of data packets comprising a reference data packet of a first data flow (404) of the at least two data flows (404, 406, 408) and one or more secondary data packets of at least a second data flow (406, 408) of the at least two data flows (404, 406, 408), wherein the secondary data packets are related to the reference data packet.FH240906PEP-2025313838. DOCX PCT24030 5155. Method for processing multi-modal data at a core network entity, the method comprising: processing at least one multi-modal data block (403_1 , 403_2), each of the at least one multi-modal data blocks (403_1 , 403_2) comprising a plurality of data packets of the at least two data flows (404, 406, 408), the plurality of data packets comprising a reference data packet of a first data flow (404) of the at least two data flows (404, 406, 408) and one or more secondary data packets of at least a second data flow (406, 408) of the at least two data flows (404, 406, 408), wherein the secondary data packets are related to the reference data packet.

56. Computer program for performing a method according to claim 53 to 55.FH240906PEP-2025313838. DOCX PCT

Images