Flow synchronization
The SEALDD client measures and adjusts traffic flow delays to synchronize multi-modal data, addressing latency issues and enhancing the immersive experience in virtual reality applications.
Patent Information
- Application Number
- PCT/IB2026/054249
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-09
- Filing Date
- 2026-04-29
- Publication Date
- 2026-11-12
AI Technical Summary
Existing multi-modal flow synchronization in immersive virtual reality applications is challenged by latency differences between traffic flows, leading to misalignment and potential packet loss, which current systems fail to address effectively.
Implementing a SEALDD client that measures and reports delay differences across multiple traffic flows, allowing for proactive alignment and quality of service adjustments to ensure synchronized delivery of multi-modal data.
Enhances synchronization of multi-modal flows, reducing latency discrepancies and packet loss, thereby improving the immersive experience in virtual reality applications.
Smart Images

Figure IB2026054249_12112026_PF_FP_ABST
Abstract
Description
FLOW SYNCHRONIZATIONFIELD
[0001] The following example embodiments relate to synchronization and especially flow synchronization in multi modal communication.BACKGROUND
[0002] Immersive multi-modal virtual reality (VR) applications or vertical applications are advanced digital environments that combine various sensory inputs to create a highly realistic and interactive experience for users. These applications utilise multiple modalities, such as visual, auditory and haptic to simulate real-world or imaginary scenarios. The goal is to immerse users so deeply that they feel as though they are physically present in the virtual environment. These applications leverage the advanced capabilities of 5G and 6G technology to enhance operations and services. These applications may be used in several different fields, such as gaming, healthcare or education, to name a few.
[0003] These applications typically require multi-modal communication which comprises multiple traffic flows that are simultaneously transmitted from a server to a client device.SUMMARY
[0004] The scope of protection sought for various example embodiments is set out by the claims. The example embodiments and features, if any, described in this specification that do not fall under the scope of the claims are to be interpreted as examples useful for understanding various embodiments.BRIEF DESCRIPTION OF THE DRAWINGS
[0005] In the following, various example embodiments will be described in greater detail with reference to the accompanying drawings, in which
[0006] FIG. 1 illustrates an example of a wireless communication network;
[0007] FIG. 2 illustrates an example of a signaling chart;
[0008] FIGS. 3, 4, 5 and 6 illustrate flow charts according to some embodiments;
[0009] FIG. 7 illustrates an example of traffic flow mismatch;
[0010] FIG. 8 is a signalling chart illustrating an embodiment; and
[0011] FIGS. 9 and 10 illustrate examples of apparatuses of some embodiments.DETAILED DESCRIPTION
[0012] The following embodiments are exemplifying. Although the specification may refer to “an”, “one”,or “some” embodiment(s) in several locations of the text, this does not necessarily mean that each reference is made to the same embodiment(s), or that a particular feature only applies to a single embodiment. Single features of different embodiments may also be combined to provide other embodiments within the scope of the claims. Furthermore, the words "comprising" and "including" should be understood as not limiting the described embodiments to consist of only those features that have been mentioned, and such embodiments may also contain features that have not been specifically mentioned. Reference numbers, in the description and / or in the claims, serve to illustrate the embodiments with reference to the drawings, without limiting the embodiments to these examples only.
[0013] Some example embodiments described herein may be implemented in a wireless communication network comprising a radio access network based on one or more of the following radio access technologies (RATs): fourth generation (4G), fifth generation (5G), 5G new radio (NR), 5G-Advanced (i.e., 3GPP NR Rel-18 and beyond), sixth generation (6G), or seventh generation (7G). Some examples of radio access networks include the universal mobile telecommunications system (UMTS) radio access network (UTRAN), the evolved universal terrestrial radio access network (E-UTRA), or the next generation radio access network (NG-RAN). The wireless communication network may further comprise a core network, and some example embodiments may also be applied to network functions of the core network.
[0014] It should be noted that the embodiments are not restricted to the wireless communication network given as an example, but a person skilled in the art may also apply the solution to other wireless communication networks or systems provided with necessary properties.
[0015] FIG. 1 depicts an example of a simplified wireless communication network showing some physical and logical entities. The connections shown in FIG. 1 may be physical connections or logical connections. It is apparent to a person skilled in the art that the wireless communication network may also comprise other physical and logical entities than those shown in FIG. 1.
[0016] The example embodiments described herein are not, however, restricted to the wireless communication network given as an example but a person skilled in the art may apply the example embodiments described herein to other wireless communication networks provided with necessary properties.
[0017] The example wireless communication network shown in FIG. 1 includes a radio access network (RAN) and a core network 110. FIG. 1 shows terminal device or user equipment (UE) 100, 102 configured to be in a wireless connection on one or more communication channels in a radio cell with an access node 104 of a radio access network. The access node 104 may comprise a computing device configuredto control the radio resources of the access node 104 and to be in a wireless connection with one or more UEs 100, 102. The access node 104 may also be referred to as a base station, a base transceiver station (BTS), an access point, a cell site, a network node, a radio access network node, a RAN node, or a network device.
[0018] The access node 104 may be, for example, an evolved NodeB (abbreviated as eNB or eNodeB), or a next generation evolved NodeB (abbreviated as ng-eNB), or a next generation NodeB (abbreviated as gNB or gNodeB), providing the radio cell. The access node 104 may include or be coupled to transceivers. From the transceivers of the access node 104, a connection may be provided to an antenna unit that establishes a bi-directional radio link to one or more UEs 100, 102. The antenna unit may comprise an antenna or antenna element, or a plurality of antennas or antenna elements.
[0019] The wireless connection (e.g., radio link) from a UE 100, 102 to the access node 104 may be called uplink (UL) or reverse link, and the wireless connection (e.g., radio link) from the access node 104 to the UE 100, 102 may be called downlink (DL) or forward link. A UE 100 may also communicate directly with another UE 102, and vice versa, via a wireless connection generally referred to as a sidelink (SL). It should be appreciated that the access node 104 or its functionalities may be implemented by using any node, host, server, access point or other entity suitable for providing such functionalities.
[0020] The radio access network may comprise more than one access node 104, in which case the access nodes may also be configured to communicate with one another over wired or wireless links. These links between access nodes may be used for sending and / or receiving control plane signaling and also for routing data from one access node to another access node.
[0021] The access node 104 may further be connected to a core network (CN) 110. The core network 110 may comprise an evolved packet core (EPC) network and / or a 5thgeneration core network (5GC). The EPC may comprise network entities, such as a serving gateway (S-GW for routing and forwarding data packets), a packet data network gateway (P-GW) for providing connectivity of UEs to external packet data networks, and / or a mobility management entity (MME). The 5GC may comprise one or more network functions, such as at least one of: a user plane function (UPF), an access and mobility management function (AMF), a location management function (LMF), and / or a session management function (SMF).
[0022] The core network 110 may also be able to communicate with one or more external networks 113, such as a public switched telephone network or the Internet, or utilize services provided by them. For example, in 5G wireless communication networks, the UPF of the core network 110 may be configured to communicate with an external data network via an N6 interface. In LTE wireless communicationnetworks, the P-GW of the core network 110 may be configured to communicate with an external data network.
[0023] It should also be understood that the distribution of functions between core network operations and access node operations may differ in future wireless communication networks compared to that of the LTE or 5G, or even be non-existent.
[0024] The illustrated UE 100, 102 is one type of an apparatus to which resources on the air interface may be allocated and assigned. The UE 100, 102 may also be called a wireless communication device, a subscriber unit, a mobile station, a remote terminal, an access terminal, a user terminal, a terminal device, or a user device, just to mention but a few names. The UE 100, 102 may be a computing device operating with or without a subscriber identification module (SIM), including, but not limited to, the following types of computing devices: a mobile phone, a smartphone, a personal digital assistant (PDA), a handset, a computing device comprising a wireless modem (e.g., an alarm or measurement device, etc.), a laptop computer, a desktop computer, a tablet, a game console, a notebook, a multimedia device, a reduced capability (RedCap) device, a wearable device (e.g., a watch, earphones or eyeglasses) with radio parts, a sensor comprising a wireless modem, or a computing device comprising a wireless modem integrated in a vehicle.
[0025] It should be appreciated that the UE 100, 102 may also be a nearly exclusive uplink-only device, of which an example may be a camera or video camera loading images or video clips to a network. The UE 100, 102 may also be a device having capability to operate in an Internet of Things (loT) network, which is a scenario in which objects may be provided with the ability to transfer data over a network without requiring human-to-human or human-to-computer interaction.
[0026] The wireless communication network may also be able to support the usage of cloud services. For example, at least part of core network operations may be carried out as a cloud service (this is depicted in FIG. 1 by “cloud” 114). The UE 100, 102 may also utilize the cloud 114. In some applications, the computation for a given UE may be carried out in the cloud 114 or in another UE.
[0027] The wireless communication network may also comprise a central control entity, such as a network management system (NMS), or the like. The NMS is a centralized suite of software and hardware used to monitor, control, and administer the network infrastructure. The NMS is responsible for a wide range of tasks such as fault management, configuration management, security management, performance management, and accounting management. The NMS enables network operators to efficiently manage and optimize network resources, ensuring that the network delivers high performance, reliability, and security.
[0028] In one embodiment, an access node 104 may comprise: a radio unit (RU) 103 comprising a radio transceiver (TRX), i.e., a transmitter (Tx) and a receiver (Rx); one or more distributed units (DUs) 105 that may be used for the so-called Layer 1 (L1) processing and real-time Layer 2 (L2) processing; and a central unit (CU) 108 (also known as a centralized unit) that may be used for non-real-time L2 and Layer 3 (L3) processing. The CU 108 may be connected to the one or more DUs 105 for example via an F1 interface. Such an embodiment of the access node 104 may enable the centralization of CUs relative to the cell sites and DUs, whereas DUs may be more distributed and may even remain at cell sites. The CU and DU together may also be referred to as baseband or a baseband unit (BBU). The CU and DU may also be comprised in a radio access point (RAP).
[0029] The CU 108 may be a logical node hosting radio resource control (RRC), service data adaptation protocol (SDAP) and / or packet data convergence protocol (PDCP), of the NR protocol stack for an access node 104. The CU 108 may comprise a control plane (CU-CP), which may be a logical node hosting the RRC and the control plane part of the PDCP protocol of the NR protocol stack for the access node 104. The CU 108 may further comprise a user plane (CU-UP), which may be a logical node hosting the user plane part of the PDCP protocol and the SDAP protocol of the CU for the access node 104.
[0030] The DU 105 may be a logical node hosting radio link control (RLC), medium access control (MAC) and / or physical (PHY) layers of the NR protocol stack for the access node 104. The operations of the DU 105 may be at least partly controlled by the CU 108. It should also be understood that the distribution of functions between the DU 105 and the CU 108 may vary depending on the implementation.
[0031] Cloud computing systems may also be used to provide the CU 108 and / or DU 105. A CU provided by a cloud computing system may be referred to as a virtualized CU (vCU). In addition to the vCU, there may also be a virtualized DU (vDU) provided by a cloud computing system. Furthermore, there may also be a combination, where the DU may be implemented on so-called bare metal solutions, for example application-specific integrated circuit (ASIC) or customer-specific standard product (CSSP) system-on-a-chip (SoC).
[0032] Edge cloud may be brought into the radio access network by utilizing network function virtualization (NFV) and software defined networking (SDN). Using edge cloud may mean access node operations to be carried out, at least partly, in a computing system operationally coupled to a remote radio head (RRH) or a radio unit (RU) 103 of an access node 104. It is also possible that access node operations may be performed on a distributed computing system or a cloud computing system located at the access node 104. Application of cloud RAN architecture enables RAN real-time functions beingcarried out at the radio access network (e.g., in a DU 105), and non-real-time functions being carried out in a centralized manner (e.g., in a CU 108).
[0033] 5G (or new radio, NR) wireless communication networks may support multiple hierarchies, where multi-access edge computing (MEC) servers may be placed between the core network 110 and the access node 104. It should be appreciated that MEC may be applied in LTE wireless communication networks as well.
[0034] A 5G wireless communication network (“5G network”) may also comprise a non-terrestrial communication network, such as a satellite communication network, to enhance or complement the coverage of the 5G radio access network. For example, satellite communication may support the transfer of data between the 5G radio access network and the core network 110, enabling more extensive network coverage. Possible use cases may include: providing service continuity for machine-to-machine (M2M) or Internet of Things (loT) devices or for passengers on board of vehicles, or ensuring service availability for critical communications, and future railway, maritime, or aeronautical communications. Satellite communication may utilize geostationary earth orbit (GEO) satellite systems, or low earth orbit (LEO) satellite systems, such as mega-constellations (i.e., systems in which hundreds of (nano)satellites are deployed). Alternatively, the satellites may be an airborne devices, such as an unmanned aerial vehicle (UAV), or a high-altitude platform system (HAPS). A given satellite 106 may provide communication services on Earth via one or more satellite beams. The one or more satellite beams create one or more cells over a given service area that may be bounded by the field of view of the satellite 106.
[0035] It is obvious for a person skilled in the art that the access node 104 depicted in FIG. 1 is just an example of a part of a radio access network, and in practice the radio access network may comprise a plurality of access nodes 104, the UEs 100, 102 may have access to a plurality of radio cells, and the radio access network may also comprise other apparatuses, such as physical layer relay access nodes or other entities. At least one of the access nodes may be a Home eNodeB or a Home gNodeB. A Home gNodeB or a Home eNodeB is a type of access node that may be used to provide indoor coverage inside a home, office, or other indoor environment.
[0036] 6G wireless communication networks are expected to adopt flexible decentralized and / or distributed computing systems and architecture and ubiquitous computing, with local spectrum licensing, spectrum sharing, infrastructure sharing, and intelligent automated management underpinned by mobile edge computing, artificial intelligence, short-packet communication and blockchain technologies. Key features of 6G may include intelligent connected management and control functions, programmability, integrated sensing and communication, reduction of energy footprint, trustworthy infrastructure,scalability and affordability. In addition to these, 6G is also targeting new use cases covering the integration of localization and sensing capabilities into system definition to the unifying user experience across physical and digital worlds.
[0037] It is envisaged that 7G, which will succeed the 5G and 6G technologies, will be able to satisfy the requirements of extremely high bandwidth, almost zero latency, and universal integration.
[0038] Some example embodiments are described below using principles and terminology of 5G radio access technology without limiting the example embodiments to 5G radio access technology, however.
[0039] The need for multi-modal flow has been increasing. This has been taken into account in the development of 5G. An architecture for handling multi-modal flows within the 5G has been created, namely Service Enabler Architecture Layer for Verticals, SEAL. Data delivery within this architecture may be denoted as SEAL Data Delivery, SEALDD. The SEAL provides a set of common services and functionalities for multi-modal flow communication within 5G.
[0040] Some functional elements within the SEAL architecture comprise- Vertical Application Layer Client, VAL client: This element provides the client-side (typically UE side) services and functionalities.- Vertical Application Layer Server (VAL server): This element provides the server-side services and functionalities.- SEAL Client: This entity provides the client-side (typically UE side) functionalities corresponding to a specific SEAL service. It operates an intermediary between VAL client and the SEAL server.- SEAL Server: This entity provides the server-side functionalities corresponding to a specific SEAL service. It takes care of requests from the SEAL client and ensures that the services are delivered efficiently. The SEAL server also manages the interaction with the underlying communication system.
[0041] In multi-modal flows comprising more than one traffic flow, synchronization between different traffic flows of a multi-modal flow, which may comprise different media components, is important. This is particularly notable when synchronization threshold between two or more traffic flows is lower than the latency Key Performance Indicator (KPI) for the application that utilizes the multi-modal flow. This emphasizes significance of minimizing delay differences between traffic flows (also referred to as flows) of a multi-modal flow. Otherwise, the arrived traffic flows of multi-modal flow need to wait for other related traffic flows of the multi-modal flow that have not yet arrived.
[0042] To avoid the downlink traffic flow delay difference introduced from (radio) access network (R)ANand Internet, SEALDD client may do multi-modal traffic flow alignment, by adjusting associated multimodal traffic flows required to arrive at application client at the same time.
[0043] FIG. 2 illustrates an example signaling chart involving an application client 200, a SEALDD client 202, a wireless communication system 201 (such as 5G network) and VAL server 204. At the VAL server, there are two associated traffic flows 210, 212. They are sent out from the VAL server 204 at the same time. However, flow 210 arrives at SEALDD client earlier than flow 212. The reason for the discrepancy in the time of arrival between flow 210 and flow 212 may be different routing paths, for example. Without flow alignment 206, the flows would arrive at the application client at different times.
[0044] To align those associated flows, the SEALDD client 202 may hold and buffer 214 the flow 210 for a while. When all the associated flows have arrived (i.e., flow 212), the SEALDD client sends out those associated flows to application client 200 at the same time. This is illustrated in 208.
[0045] However, although SEALDD clients may support the feature of holding the flow packets which arrived early till all other associated flows arrived, it may still be possible that the flows are arriving at different time due to issues that arise during the transporting of the flows. The core network or the SEALDD server are not aware of the possible situation where flows are arriving at different times at SEALDD client. Thus, the core network and / or the SEALDD server are not able to perform any course corrections to ensure that the flows will arrive at the same time in the future. Further, if the information about the flow alignment is not provided to the core network or SEALDD server, the problem of associated flows arriving at different times may occur frequently and at some point, traffic flow packets may not arrive at the SEALDD client (i.e. UE) at all.
[0046] FIG. 3 illustrates a flow chart according to an example embodiment. The flow chart illustrates the operation of a client device such as a terminal device, user terminal or UE. The device may be running of a SEALDD client application.
[0047] In step 300, the apparatus is configured to receive, from a server (e.g., a Service Enabler Architecture Layer for Verticals Data Delivery server) of a wireless communication network, a request to provide a report including information related to a delay difference measurement associated with a multimodal flow comprising two or more traffic flows received via the wireless communication network, the request comprising one or more parameters describing the report that is to be provided.
[0048] In step 302, the apparatus is configured to determine, based on the one or more parameters of the request, the delay difference measurement associated with the multi-modal flow comprising the two or more traffic flows received from the wireless communication network.
[0049] In step 304, the apparatus is configured to transmit, to the server, a measurement report generated according to the one or more parameters.
[0050] FIG. 4 illustrates a flow chart according to an example embodiment. The flow chart of FIG. 4 illustrates the operation of a network apparatus The apparatus may be running of a SEALDD server application.
[0051] In step 400, the apparatus is configured to communicate with a user equipment via a wireless communication system.
[0052] In step 402, the apparatus is configured to receive, from a server, a request to provide a report including information related to a delay difference measurement associated with a multi-modal flow between the server and the user equipment via the wireless communication system, the multi-modal flow comprising two or more traffic flows, the request comprising one or more parameters describing the report that is to be provided.
[0053] In step 404, the apparatus is configured to transmit, based on the receiving, the request to the user equipment.
[0054] In step 406, the apparatus is configured to determine, based on the one or more parameters of the request, the delay difference measurement related to delay difference between the two or more traffic flows of the multi-modal flow received from the server at the apparatus.
[0055] In step 408, the apparatus is configured to receive, from the user equipment, a measurement report based on the request transmitted to the user equipment.
[0056] In step 410, the apparatus is configured to determine, based on the delay difference measurement that is determined by apparatus and the measurement report that is received, that there is a delay difference due to either the server or the wireless communication system, based on the performed measurements and the received measurement report, that there is a delay difference due to either the server or the communication system.
[0057] In step 412, the apparatus is configured to notify, based on the determining, at least one of the server or the wireless communication system about the delay difference.
[0058] FIG. 5 illustrates a flow chart according to an example embodiment. The flow chart illustrates the operation of a client device such as a terminal device, user equipment, or user terminal. The device may be running of a SEALDD client application.
[0059] In step 500, the apparatus is configured to receive, from a server (e.g. a SEAL Data Delivery server) of a wireless communication network, a request to report information about measurements ofdelays associated with a multi-modal flow received from the network, the multi-modal flow comprising traffic flows.
[0060] In step 502, the apparatus is configured to perform, based on the request, the measurements of the delays of the traffic flows of the multi-modal flow received from the network.
[0061] In step 504, the apparatus is configured to determine, based on the measurements of the delays, one or more identifications of one or more traffic flows of the multi-modal flow that the user equipment did not receive within a delay threshold.
[0062] In step 506, the apparatus is configured to transmit, to the server, a measurement report comprising the one or more identifications of the one or more traffic flows.
[0063] FIG. 6 illustrates a flow chart according to an example embodiment. The flow chart illustrates the operation of a network apparatus The apparatus may be running of a SEALDD server application.
[0064] In step 600, the apparatus is configured to communicate via a communication system with a client device.
[0065] In step 602, the apparatus is configured to receive, from a server (e.g. a Vertical Application Layer server, a request to provide a report including information related to delay measurements associated with a multi-modal flow between the server and a user equipment via the wireless communication system, the multi-modal flow comprising traffic flows.
[0066] In step 604, the apparatus is configured to transmit, based on the receiving, the request to the user equipment.
[0067] In step 606, the apparatus is configured to measure, in response to the request, delays of the traffic flows of the multi-modal flow received from the server at the apparatus.
[0068] In step 608, the apparatus is configured to receive, from the user equipment the measurement report based on the request transmitted to the user equipment, the measurement report comprising identifications of traffic flows that the client did not receive within a delay threshold.
[0069] In step 610, the apparatus is configured to determine, based on the measurements and the measurement report that is received, that there is a delay due to either the server or the wireless communication system.
[0070] In step 612, the apparatus is configured notify, based on the determining, at least one of the server or the communication system about the delay and the identifications of traffic flows.
[0071] Thus, in an embodiment, a terminal device running SEALDD client receives a request from the network element acting as a SEALDD Server to measure and report delay difference measurements of traffic flows associated with a multi-modal flow in a measurement report. The SEALDD Server may alsobe configured to measure corresponding delay differences. The SEALDD Server will receive measurements of made by the SEALDD client. In an embodiment, the SEALDD client transmits a measurement report to the network the measurement report is to be generated and transmitted when at least one of the following occurs: a maximum delay difference measurement of the delay difference measurement between traffic flows of the two or more traffic flows of the multi-modal flow exceeds a threshold; or an expiration of a time period.
[0072] In an embodiment, the VAL server may request the SEALDD Server to start monitoring and reporting delay measurements of multi-modal flows.
[0073] Based on the measurements, the SEALDD Server may detect if there are delay differences between the traffic flows and further determine where these delay differences may occur, whether the differences are due to the VAL server or the communication system. Based on the determination, actions to correct the situation may be taken.
[0074] In an embodiment, the report is to comprise one or more of the following: minimum delay difference between received traffic flows of the multi-modal flow, maximum delay difference between received traffic flows of the multi-modal flow, average delay difference between all received traffic flows of the multi-modal flow, identification of the flow which arrived last to the apparatus, identification of the flow which arrived first to the apparatus and time stamp of each received traffic flows of the multi-modal flow.
[0075] In an embodiment, the given threshold is received from the network. The given threshold may be the maximum acceptable duration for traffic flow alignment of the network minus a triggering criterion. The apparatus may be configured to receive the maximum acceptable duration for traffic flow alignment of the network and the triggering criterion from the network.
[0076] Further, in an embodiment, a terminal device running SEALDD client receives a request from the network element acting as a SEALDD Server to measure delay measurements of traffic flows associated with a multi-modal flow and determine and report identifications of traffic flows that the apparatus received later than a given delay threshold or did not receive at all.
[0077] In an embodiment, the request from the network element comprises the triggering criterion. The measurement parameter exceeding the given threshold may be the maximum delay difference between received traffic flows of the multi-modal flow.
[0078] In an embodiment, SEALDD Server, after receiving the report and performing its own measurements, determines that the delay difference is due to the caused by the communication system,if the delay measured by the SEALDD client is greater than the sum of the delay difference measured by the SEALDD Server and the quality of service guaranteed by the communication system.
[0079] In an embodiment, SEALDD Server, after receiving the report and performing its own measurements, determines that the delay difference is due to / caused by the connection between the apparatus and the server, if the delay measured by the SEALDD client is smaller or equal than the sum of the delay difference measured by the SEALDD Server and the quality of servicer guaranteed by the communication system.
[0080] In an embodiment, based on performed measurements, the SEALDD client may determine identifications of traffic flows that the SEALDD client received later than a given delay threshold or did not receive at all and transmit a measurement report comprising the identifications of the traffic flows to the SEALDD Server. The given delay threshold may be a maximum delay threshold or a synchronization threshold delay, for example. The SEALDD client may create a list of the identifications of traffic flows that the apparatus received later than a given delay threshold or did not receive at all and include the list in the measurement report.
[0081] The proposed solution enables estimating the performance of applications utilising multi-modal flows and benchmark the performance of the applications. In addition, based on the proposed solution it is possible to isolate the transmission path segment that is introducing delay. For example, it is possible to determine whether delay is due to transport path between SEALDD Client <-> Core network <-> SEALDD Server o SEALDD Server <-> VAL Server so that corrective actions can be taken accordingly.
[0082] Further, multi modal synchronization misalignment imposes storage requirement at SEAL Client (typically user terminal or UE). By measuring the delay differences across associated flows, data storage requirements at the SEAL Client can be controlled effectively.
[0083] FIG. 7 illustrates an example situation of traffic flow mismatch in communication related to a multi-modal flow 700. FIG 7 illustrates flow traffic between SEALDD Server 702, a wireless communication system 201 (such as 5G network) and a SEALDD client 202. In this example, the multimodal flow 700 comprises six traffic flows which are transmitted by the SAED server 7802 at the same time via the communication system to the SEALDD client 202. The traffic flows arrive at the SEALDD client at different times. Flows 1 and 2 arrive at time instant TO, flow 3 arrives at time instant T1, flow 4 arrives at time instant T2, flow 5 arrives at time instant T3 and flow 6 arrives at time instant T4.
[0084] SEALDD client 202 may be configured to measure one or more delay parameters of the traffic flows. In this example, the SEALDD client may determine that measured maximum delay difference is T4-T0, minimum delay difference is T1-T0, average delay difference ((T1-T0)+ (T2-T0)+ (T3-T0)+(T4-T0)) / 6.
[0085] In addition, SEALDD client 202 may be configured to compare the delays of the traffic flows to a given synchronization threshold or maximum delay threshold limit 706.
[0086] In an embodiment, following information elements may be used to define proposed requests and reports.
[0087] In an embodiment, transmission quality measurement subscription request may be defined as:Information element Status Description> Measurement ID M Measurement identifiers, e.g. latency, bitrate, jitter, signal strength (e.g. RSRP, SI NR, RSRQ, RSSI), Multi modal flow delay measurement information (multi modal flowdelay difference, Exceeded flow id list. )In an embodiment, transmission quality measurement subscription request may be defined as:Information element Status DescriptionMulti modal flow delay O Minimum, Average multi modal flow delay measurement information differenceExceeded flow ID list.Frequency at which SEALDD client should send the report to SEALDD server.> Measurement ID M Measurement identifiers, e.g. UL / DL / E2E latency, UL / DL bitrate, UL / DL / E2E packet loss rate, UL / DL / E2E jitter, and crossflow identifier (e.g., crossflow latency, crossflow bitrate, crossflow packet loss rate, crossflow jitter), Multi modal flow delay measurementinformation.
[0088] In an embodiment, SEALDD transmission quality measurement reports list may be defined as:Information element Status Description> Measurement ID M Measurement identifiers, (e.g. UL / DL / E2E latency, UL / DL bitrate, UL / DL / E2E packet loss rate, UL / DL / E2E jitter) and crossflow identifiers (e.g., crossflow latency, crossflow bitrate, crossflow packet loss rate, crossflow jitter), Multi modal flow delay measurementinformation.
[0089] FIG. 8 is a signalling chart illustrating an embodiment. FIG 8 illustrates an example of operations of VAL Client 800, SEALDD Client 202, wireless communication system 201 (such as 5G network), SEALDD Server 702 and a VAL server 802.
[0090] In 804, an on-going data transmission connection is established between the VAL server 802 and VAL client 800.
[0091] The VAL server 802 transmits a SEALDD transmission quality measurement subscription request 806 to the SEALDD server 702. In addition to already supported prior art measurement IDs, the request includes multi modal delay difference measurement and frequency of the report. After authorization, the SEALDD Server 702 transmits a response to VAL Server (not shown).
[0092] Then the SEALDD server forwards 808 the transmission quality measurement request to the SEALDD Client 202 via the communication system 201. In addition to already prior art supported measurement IDs, the request includes multi modal delay difference measurement. The SEALDD client acknowledges response to SEALDD server (not shown).
[0093] The SEALDD Client 202 and SEALDD server 702 derive and apply 810, 812 flow alignment assistant information. When a multi-modal flows alignment policy is triggered, the SEALDD server 702 may help to provide the flows alignment assistance information (such as timestamp in the Real-time Transport Protocol, RTP, header, RTP Control Protocol, RTCP). If the maximum acceptable duration for traffic flow alignment is not provided, then the SEALDD server may determine the maximum acceptable duration for traffic flow alignment based on VAL service ID, flows transmission requirement, transmission quality, and the synchronization threshold. The server may translate the traffic descriptor into multi-modal SEALDD flow ID.
[0094] Then, the downlink multi modal flow data transmission 814, 816 is in progress.
[0095] The SEALDD Client and SEALDD server measure 818, 820 one or more parameters of the associated flow delay differences. In an embodiment, they calculate minimum, average and maximum delay difference and determine the flows exceeding given threshold.
[0096] The SEALDD client transmits 822 transmission quality measurement report that includes measured multi modal flow delay difference measurements to the SEALDD server. In an embodiment, the measurement report is transmitted if a measurement parameter exceeds a given threshold.
[0097] The SEALDD Server transmits 824 the transmission quality measurement report that includes measured multi modal flow delay difference measurements to VAL Server.
[0098] The SEALDD server derives 826 the current multi modal flow alignment delay situation based on received measurements.
[0099] In case the delay recorded at SEALDD Client is greater than the delay recorded at SEALDD Server added with the quality of service, QoS, guaranteed by the communication system, the SEALDD server determines that the multi modal flow misalignment is due to additional delay caused by thecommunication system not meeting the guaranteed QoS. The SEALDD server then notifies 828 the communication system to adjust 832 QoS profiles and relevant routing policies of associated flows.
[0100] In case the delay recorded at SELDD Client is smaller or equal than the delay recorded at SEALDD Server added with the QoS guaranteed by the communication system, the SEALDD server determines that the multi modal flow misalignment at SEALDD client is due to flows not arriving in expected time from the VAL server 802. The SEALDD server then notifies 830 the VAL Server to adjust 834 associated flow(s) flow control.
[0101] In an embodiment, the SEALDD Server is configured to interact with the core network of the communication system to adjust quality of service settings of the communication system.
[0102] In an embodiment, the SEALDD Server is configured to receive from the server adjusted synchronization threshold delay.
[0103] In an embodiment, the SEALDD Server is configured to provide flow alignment information to the communication system for proactive adjustments to improve synchronisation.
[0104] In an embodiment, the SEALDD Server is configured to perform packet encapsulation with timestamp information when transmitting to the client device and calculate transmission delay measurement result of traffic flows after receiving timestamp from the client device.
[0105] FIG. 9 illustrates an example of an apparatus 900 comprising means for causing the apparatus 900 to perform one or more of the example embodiments (e.g., the methods of FIG. 3 or FIG. 5) described above. For example, the apparatus 900 may be an apparatus such as, or comprising, or comprised in, a user equipment (UE) 100, 102. The user equipment may also be called a wireless communication device, a subscriber unit, a mobile station, a remote terminal, an access terminal, a user terminal, a terminal device, or a user device. The apparatus may also be a VR unit, such as a wearable VR device.
[0106] The apparatus 900 may comprise a circuitry or a chipset applicable for realizing one or more of the example embodiments described above. For example, the apparatus 900 may comprise at least one processor 910. The at least one processor 910 interprets instructions (e.g., computer program instructions) and processes data. The at least one processor 910 may comprise one or more programmable processors. The at least one processor 910 may comprise programmable hardware with embedded firmware and may, alternatively or additionally, comprise one or more application-specific integrated circuits (ASICs).
[0107] The at least one processor 910 is coupled to at least one memory 920. The at least one processor is configured to read and write data to and from the at least one memory 920. The at least one memory 920 may comprise one or more memory units. The memory units may be volatile or non-volatile. It is tobe noted that there may be one or more units of non-volatile memory and one or more units of volatile memory or, alternatively, one or more units of non-volatile memory, or, alternatively, one or more units of volatile memory. Volatile memory may be for example random-access memory (RAM), dynamic randomaccess memory (DRAM) or synchronous dynamic random-access memory (SDRAM). Non-volatile memory may be for example read-only memory (ROM), programmable read-only memory (PROM), electronically erasable programmable read-only memory (EEPROM), flash memory, optical storage or magnetic storage. In general, memories may be referred to as non-transitory computer readable media. The term “non-transitory,” as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., RAM vs. ROM). The at least one memory 920 stores computer readable instructions 930 that are executed by the at least one processor 910 to perform one or more of the example embodiments described above. For example, non-volatile memory stores the computer readable instructions, and the at least one processor 910 executes the instructions using volatile memory for temporary storage of data and / or instructions. The computer readable instructions may refer to computer program code.
[0108] The computer readable instructions may have been pre-stored to the at least one memory 920 or, alternatively or additionally, they may be received, by the apparatus, via an electromagnetic carrier signal and / or may be copied from a physical entity such as a computer program product. Execution of the computer readable instructions by the at least one processor 910 causes the apparatus 900 to perform one or more of the example embodiments described above. That is, the at least one processor and the at least one memory storing the instructions may provide the means for providing or causing the performance of any of the methods and / or blocks described above.
[0109] In the context of this document, a “memory” or “computer-readable media” or “computer-readable medium” may be any non-transitory media or medium or means that can contain, store, communicate, propagate or transport the instructions for use by or in connection with an instruction execution system, apparatus, or device, such as a computer. The term “non-transitory,” as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., RAM vs. ROM).[O11O] The apparatus 900 may further comprise, or be connected to, an input unit 940. The input unit 940 may comprise one or more interfaces for receiving input. The one or more interfaces may comprise, for example, at least one of: one or more cameras, one or more accelerometers, one or more microphones, one or more buttons and / or one or more touch detection units. Further, the input unit 940 may comprise an interface to which external devices may connect to.
[0111] The apparatus 900 may also comprise an output unit 950. The output unit may comprise or be connected to one or more displays capable of rendering visual content, such as a light emitting diode (LED) display, a liquid crystal display (LCD) and / or a liquid crystal on silicon (LCoS) display. The output unit 950 may further comprise one or more audio outputs. The one or more audio outputs may be for example loudspeakers.
[0112] The apparatus 900 may further comprise or be connected to a communication interface 960, such as a radio unit, comprising hardware and / or software for realizing communication connectivity with one or more wired or wireless communication devices according to one or more communication protocols. The communication interface 930 comprises at least one transmitter (Tx) and at least one receiver (Rx) that may be integrated to the apparatus 900 or that the apparatus 900 may be connected to. The communication interface 930 may provide means for performing some of the blocks and / or functions (e.g., transmitting and receiving) for one or more example embodiments described above. The communication interface 930 may comprise one or more components, such as: power amplifier, digital front end (DFE), analog-to-digital converter (ADC), digital-to-analog converter (DAC), frequency converter, (de)modulator, and / or encoder / decoder circuitries, controlled by the corresponding controlling units.
[0113] The communication interface 930 may provide the apparatus with radio communication capabilities to communicate in the wireless communication network, for example. The communication interface may, for example, provide a radio interface to one or more UEs 100, 102.
[0114] It is to be noted that the apparatus 900 may further comprise various components not illustrated in FIG. 9. The various components may be hardware components and / or software components.
[0115] FIG. 10 illustrates an example of an apparatus 1000 comprising means for causing the apparatus 1000 to perform one or more of the example embodiments (e.g., the methods of FIG. 4 and FIG. 6) described above. For example, the apparatus 1000 may be a network element or server.
[0116] The apparatus 1000 may comprise, for example, a circuitry or a chipset applicable for realizing one or more of the example embodiments described above. The apparatus 1000 may be an electronic device comprising one or more electronic circuitries. The apparatus 1000 may comprise a communication control circuitry 1010 such as at least one processor, and at least one memory 1020 storing instructions 1025 which, when executed by the at least one processor, cause the apparatus 1000 to carry out one or more of the example embodiments described above. Such instructions 1022 may, for example, include computer program code (software). The at least one processor and the at least one memory storing the instructions may provide the means for providing or causing the performance of any of the methodsand / or blocks described above.
[0117] The processor is coupled to the memory 1020. The processor is configured to read and write data to and from the memory 1020. The memory 1020 may comprise one or more memory units. The memory units may be as described above in connection with FIG. 9.
[0118] The computer readable instructions may have been pre-stored to the memory 1020 or, alternatively or additionally, they may be received, by the apparatus, via an electromagnetic carrier signal and / or may be copied from a physical entity such as a computer program product. Execution of the computer readable instructions causes the apparatus 1000 to perform one or more of the functionalities described above.
[0119] The memory 1020 may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, flash memory, magnetic memory devices and systems, optical memory devices and systems, fixed memory and / or removable memory. The memory may comprise a configuration database for storing configuration data, such as a current neighbour cell list, and, in some example embodiments, structures of frames used in the detected neighbour cells.
[0120] The apparatus 1000 may further comprise a connectivity unit 1030. The connectivity unit 1030 enables wired or wireless connectivity to one or more external devices. The connectivity unit 1030 comprises at least one transmitter and at least one receiver that may be integrated to the apparatus 100 or that the apparatus 1000 may be connected to.
[0121] In an embodiment, the at least one transmitter comprises at least one transmission antenna, and the at least one receiver comprises at least one receiving antenna. The connectivity unit 1030 may comprise an integrated circuit or a set of integrated circuits that provide wired and / or wireless communication capability for the apparatus 1000. Alternatively, the wireless connectivity may be a hardwired application-specific integrated circuit (ASIC). The connectivity unit 1030 may also provide means for performing at least some of the blocks or functions of one or more example embodiments described above. The connectivity unit 1030 may comprise one or more components, such as: power amplifier, digital front end (DFE), analog-to-digital converter (ADC), digital-to-analog converter (DAC), frequency converter, (de)modulator, and / or encoder / decoder circuitries, controlled by the corresponding controlling units.
[0122] In an embodiment, the apparatus 1000 may further comprise, or be connected to, an input / output unit 1040. The input unit 1040 may comprise one or more interfaces for receiving input. The one or more interfaces may comprise, for example, at least one of: one or more cameras, one or more accelerometers, one or more microphones, one or more buttons and / or one or more touch detection units. Further, theinput unit 1040 may comprise an interface to which external devices may connect to.
[0123] The apparatus 1000 may also comprise an output unit 1050. The output unit may comprise or be connected to one or more displays capable of rendering visual content, such as a light emitting diode (LED) display, a liquid crystal display (LCD) and / or a liquid crystal on silicon (LCoS) display. The output unit 1050 may further comprise one or more audio outputs. The one or more audio outputs may be for example loudspeakers.
[0124] It is to be noted that the apparatus 1000 may further comprise various components not illustrated in FIG. 10. The various components may be hardware components and / or software components.
[0125] As used in this application, the term “circuitry” may refer to one or more or all of the following: a) hardware-only circuit implementations (such as implementations in analog, digital and / or quantum circuitry); and b) combinations of hardware circuit(s) and software, such as (as applicable): i) a combination of analog, digital and / or quantum hardware circuit(s) with software / firmware and ii) any or all portions of hardware processor(s) (including digital and / or quantum processor(s)) with software, and memory(ies) that work together to cause an apparatus, such as a mobile device, computing device, or server, to perform various functions; and c) any or all portions of hardware circuit(s), such as microprocessor(s), processor(s) and / or quantum processor(s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.
[0126] This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.
[0127] The techniques and methods described herein may be implemented by various means. For example, these techniques may be implemented in hardware (one or more devices), firmware (one or more devices), software (one or more modules), or combinations thereof. For a hardware implementation, the apparatus(es) of example embodiments may be implemented within one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), graphics processing units (GPUs), processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof. For firmware or software, theimplementation can be carried out through modules of at least one chipset (for example procedures, functions, and so on) that perform the functions described herein. The software codes may be stored in a memory unit and executed by processors. The memory unit may be implemented within the processor or externally to the processor. In the latter case, it can be communicatively coupled to the processor via various means, as is known in the art. Additionally, the components of the systems described herein may be rearranged and / or complemented by additional components in order to facilitate the achievements of the various aspects, etc., described with regard thereto, and they are not limited to the precise configurations set forth in the given figures, as will be appreciated by one skilled in the art.
[0128] It will be understandable to a person skilled in the art that, as technology advances, the proposed concept may be implemented in various ways within the scope of the claims. The embodiments are not limited to the example embodiments described above, but may vary within the scope of the claims. Therefore, all words and expressions should be interpreted broadly, and they are intended to illustrate, not to restrict, the embodiments.
Claims
CLAIMS1. A user equipment comprising at least one processor, and at least one memory storing instructions which, when executed by the at least one processor, cause the user equipment at least to perform:receiving, from a server of a wireless communication network, a request to report information about measurements of delays associated with a multi-modal flow received from the network, the multi-modal flow comprising traffic flows;performing, based on the request, the measurements of the delays of the traffic flows of the multi-modal flow received from the network;determining, based on the measurements of the delays, one or more identifications of one or more traffic flows of the multi-modal flow that the user equipment did not receive within a delay threshold;transmitting, to the server, a measurement report comprising the one or more identifications of the one or more traffic flows.
2. The user equipment according to claim 1, wherein delay threshold is a maximum delay threshold.
3. The user equipment according to claim 1 , wherein delay threshold is a synchronization threshold delay.
4. The user equipment according to any preceding claim, wherein the instructions, when executed by the at least one processor, cause the user equipment to perform:creating a list of one or more identifications of one or more traffic flows of the multi-modal flow that the apparatus received that the user equipment did not receive within a delay threshold; and, wherein the measurement report includes the list, the one or more identifications of the measurement report comprised in the list.
5. The user equipment according to any preceding claim, wherein the delay threshold is received from the network.
6. The user equipment according to any preceding claim, wherein the instructions, when executed by the at least one processor, cause the user equipment to perform:buffering, at least until all the traffic flows of the multi-modal flow have arrived at the user equipment, traffic flows of the multi-modal flow that arrive at the user equipment earlier than other traffic flows of the multi-modal flow.
7. The user equipment according to any preceding claim, wherein the instructions, when executed by the at least one processor, cause the user equipment to perform:buffering one or more traffic flow of the multi-modal flow that arrive at the user equipment at least until all the traffic flows of the multi-modal flow have arrived at the user equipment.
8. An apparatus for a wireless communication system, the apparatus comprising at least one processor, and at least one memory storing instructions which, when executed by the at least one processor, cause the apparatus at least to perform:communicating via the communication system with a client device;receiving, from a server, a request to provide a report including information related to delay measurements associated with a multi-modal flow between the server and a user equipment via the wireless communication system, the multi-modal flow comprising traffic flows;transmitting, based on the receiving, the request to the user equipment;measure, in response to the request, delays of the traffic flows of the multi-modal flow received from the server at the apparatus;determine, based on the request, identifications of traffic flows of the multi-modal flow that the apparatus did not receive within a delay threshold;receiving, from the user equipment the measurement report based on the request transmitted to the user equipment, the measurement report comprising identifications of traffic flows that the client did not receive within a delay threshold;determine, based on the measurements and the measurement report that is received, that there is a delay due to either the server or the wireless communication system; andnotify, based on the determining, at least one of the server or the communication system about the delay and the identifications of traffic flows.
9. The apparatus according to claim 8, wherein delay threshold is a maximum delay threshold.
10. The apparatus according to claim 8, wherein delay threshold is a synchronization threshold delay.
11. The apparatus according to any one of claim 8 to claim 10, wherein the measurement report comprises a list of the identifications of traffic flows that the user equipment did not receive within the delay threshold,12. The apparatus according to any one of claim 8 to claim 11, wherein the instructions, when executed by the at least one processor, cause the apparatus to perform:performing, when the delay of at least one traffic flow of the multi-modal flow is caused by the wireless communication system, interactions with the wireless communication system to adjust quality of service settings of the wireless communication system.
13. The apparatus according to any one of claim 8 to claim 12, wherein the instructions, when executed by the at least one processor, cause the apparatus to perform:receiving, from the server, an updated of the threshold delay, the threshold delay comprises a synchronization threshold delay.
14. The apparatus according to any one of claim 8 to claim 13, wherein the instructions, when executed by the at least one processor, cause the apparatus to perform:determining a delay of at least one traffic flow of the multi-modal flow is caused by the wireless communication system in an instance a delay of at least one traffic flow of the multi-modal flow measured by the user equipment and included in the measurement report is greater than the sum of a delay of at least one traffic flow of the multi-modal flow measured by the apparatus and a quality of service guaranteed by the communication system.
15. The apparatus according to any one of claim 10 to claim 14, wherein the instructions, when executed by the at least one processor, cause the apparatus to perform:providing, to the wireless communication system, based on the determining the delay is caused by the wireless communication system, information related to flow alignment, in order for the wireless communication system to adjust at least one of a respective quality of service profile or a respective routing policy associated with at least one respective traffic flow of the two or more traffic flows of the multi-modal flow.
16. The apparatus according to claim 10 to claim 15, wherein the instructions, when executed by the at least one processor, cause the apparatus to perform:determining the delay is caused by a connection between the apparatus and the server in an instance a delay of at least one traffic flow of the multi-modal flow by the user equipment and included in the measurement report is smaller than or equal to a sum of a delay of at least one traffic flow of the multi-modal flow measured by the apparatus and a quality of service guaranteed by the wireless communication system.
17. A non-transitory computer readable medium comprising program instructions which, when executed by an apparatus, cause the apparatus to perform at least the following:communicating via the communication system with a client device;receiving, from a server, a request to provide a report including information related to delay measurements associated with a multi-modal flow between the server and a user equipment via the wireless communication system, the multi-modal flow comprising traffic flows;transmitting, based on the receiving, the request to the user equipment;measure, in response to the request, delays of the traffic flows of the multi-modal flow received from the server at the apparatus;determine, based on the request, identifications of traffic flows of the multi-modal flow that the apparatus did not receive within a delay threshold;receiving, from the user equipment the measurement report based on the request transmitted to the user equipment, the measurement report comprising identifications of traffic flows that the client did not receive within a delay threshold;determine, based on the measurements and the measurement report that is received, that there is a delay due to either the server or the wireless communication system; andnotify, based on the determining, at least one of the server or the communication system about the delay and the identifications of traffic flows.
