Data synchronization method and device

Abstract

The invention provides a data synchronization method and related equipment, and the method comprises the steps: obtaining the time delay of a first data flow relative to a second data flow in a multi-mode service, and transmitting first information to network equipment when the time delay of the first data flow relative to the second data flow is greater than or equal to a threshold value, the first information is used for indicating that the time delay of the first data stream relative to the second data stream is greater than or equal to a threshold value, then receiving second information sent by the network device, the second information comprising resource configuration information for the first data stream and / or the second data stream, and then according to the second information, sending the first data stream to the network device. And determining a resource for transmitting the first data stream and / or a resource for transmitting the second data stream. In the method, the terminal can monitor the time delay of different data streams in the multi-mode service, and when the time delay is greater than or equal to the threshold value, the network equipment can be notified to configure the resources of the data streams, so that the synchronism of the different data streams is ensured, the service requirements are met, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the field of communication technologies, and in particular, to a data synchronization method, an electronic device, a communication system, and a computer-readable storage medium. Background Art

[0002] With the development of mobile communication technologies, especially the continuous development of new-generation mobile communication technologies such as the fifth generation mobile networks (abbreviated as 5G), the functions of communication systems are constantly being enhanced. Specifically, the 5G communication system can provide enhanced Mobile Broadband (eMBB), with faster connections, higher throughput, and greater capacity, and provide ultra-reliable low-latency communications (uRLLC), so as to apply the network to critical mission scenarios that require uninterrupted and stable data links, such as Extended Reality (XR) scenarios or Cloud Gaming scenarios, to meet the requirements of the scenarios for the ultra-high reliability and low latency of wireless communication networks.

[0003] The services in the above scenarios may include Multi-modal Services. Among them, multi-modal services refer to services that transmit data streams of multiple modalities, and the multiple modalities can be any combination of modalities such as audio, video, and touch. Taking the XR scenario as an example, XR services usually need to transmit data streams including audio, video, and touch.

[0004] Currently, the network side usually adopts an independent scheduling scheme for different data streams, which may result in a relatively large time delay of one data stream relative to another in a multi-modal service scenario. For example, the video data stream arrives a long time after the audio data stream arrives, and it is difficult to synchronize different data streams. Summary of the Invention

[0005] This application provides a data synchronization method and related devices, aiming to solve the problem that the network side adopts an independent scheduling scheme for different data streams, which may result in a relatively large time delay of one data stream relative to another in a multi-modal service scenario, and it is difficult to synchronize different data streams.

[0006] To achieve the above object, this application provides the following technical solutions:

[0007] The first aspect of the present application provides a data synchronization method. This method can be executed by a terminal. A terminal, also known as a terminal device, user unit, user equipment (UE), user station, mobile station (MS), mobile device, user terminal, wireless communication device, user agent, or user device. A terminal may include, but is not limited to, a mobile phone, a tablet (pad), a computer with wireless transceiver function, a virtual reality (VR) terminal, and an augmented reality (AR) terminal.

[0008] Specifically, the terminal may obtain the delay of the first data stream relative to the second data stream in the multimodal service. When the delay of the first data stream relative to the second data stream is greater than or equal to a threshold, the terminal sends a first message to the network device. The first message is used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to the threshold. Then the terminal receives a second message sent by the network device, and the second message includes resource configuration information for the first data stream and / or the second data stream. The terminal may determine the resources for transmitting the first data stream and / or the resources for the second data stream according to the second message.

[0009] In this method, the terminal can monitor the delays of different data streams in the multimodal service. When the delay is greater than or equal to the threshold, the network device can be notified to configure the resources of the data stream to ensure the synchronization of different data streams, meet the service requirements, and improve the user experience.

[0010] In some possible implementation manners, the first message includes:

[0011] The identifier of the first data stream and the identifier of the second data stream; or,

[0012] At least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the identifier of the first data stream, the identifier of the second data stream; or,

[0013] The identifier of the first logical channel and the identifier of the second logical channel, where the first logical channel is used to transmit the first data stream, and the second logical channel is used to transmit the second data stream; or,

[0014] and at least one of the time delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, and the rate difference between the first data stream and the second data stream, and the identifier of the first logical channel, the identifier of the second logical channel; or,

[0015] the identifier of the first data radio bearer (DRB) and the identifier of the second DRB, where the first DRB is used to transmit the first data stream and the second DRB is used to transmit the second data stream; or,

[0016] at least one of the time delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, and the rate difference between the first data stream and the second data stream, and the identifier of the first DRB, the identifier of the second DRB; or,

[0017] the mode of the first data stream and the mode of the second data stream; or,

[0018] at least one of the time delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, and the rate difference between the first data stream and the second data stream, and the mode of the first data stream, the mode of the second data stream; or,

[0019] an enumerated value, where the enumerated value is used to indicate that the time delay of the first data stream relative to the second data stream is greater than or equal to a threshold; or,

[0020] at least one of the time delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, and the rate difference between the first data stream and the second data stream, and the enumerated value, where the enumerated value is used to indicate that the time delay of the first data stream relative to the second data stream is greater than or equal to a threshold.

[0021] In this method, the terminal can indicate that the time delay is greater than or equal to the threshold in various ways such as the identifier of the data stream, the identifier of the logical channel, the identifier of the DRB, the mode of the data stream, and the enumerated value, which has good flexibility and improves usability. Further, when indicating that the time delay is greater than or equal to the threshold, at least one of the specific time delay, the buffer size, rate, buffer difference, or rate difference of the data streams being compared (such as the first data stream and the second data stream) can also be indicated, so that more abundant information can be provided for resource allocation on the network side.

[0022] In some possible implementations, the terminal may send user equipment assistance information (UAI) to the network device, where the UAI includes first information. Alternatively, the terminal may send a media access control layer control element (MAC CE) to the network device, where the MAC CE includes the first information. Alternatively, the terminal may send a packet data convergence protocol (PDCP) layer control message to the network device, where the PDCP layer control message includes the first information.

[0023] In this method, the terminal can send the first information to the network side in different forms such as user equipment assistance information, MAC CE, or PDCP layer control messages, which has high flexibility and improves the overall usability.

[0024] In some possible implementations, the UAI includes a flow information structure, and the flow information structure includes synchronization information for a multi-modal service. The synchronization information is an array including the identifier of a first data stream and the identifier of a second data stream. By multiplexing the flow information structure, for example, the synchronization information in the flow information structure, to carry the first information, this method can reduce the impact on other services and also reduce the transmission overhead of sending the first information.

[0025] In some possible implementations, the array further includes at least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, and the rate difference between the first data stream and the second data stream. By adding the delay of the first data stream relative to the second data stream to the array of synchronization information, this method can provide richer information for network-side resource configuration, enabling the network side to better configure resources.

[0026] In some possible implementations, the MAC CE includes the identifier of a first logical channel and the identifier of a second logical channel. By using the MAC CE to send the identifier of the first logical channel and the identifier of the second logical channel, this method can enable the network side to sense that the delay of the first data stream relative to the second data stream is greater than a threshold, and the network side can perform resource configuration to ensure the synchronization of the data streams.

[0027] In some possible implementations, the PDCP layer control message includes a type field and an identifier field. The type field is used to indicate an enumerated value of synchronization information, and the identifier field is used to indicate the identifier of a first data radio bearer (DRB) and the identifier of a second DRB. This can enable the network side to sense that the delay of the first data stream relative to the second data stream is greater than a threshold, and the network side can perform resource configuration to ensure the synchronization of the data streams.

[0028] In some possible implementation manners, considering the capability differences of different terminals, the terminal may further send the capability information of the terminal to the network device, where the capability information is used to indicate the support capability of the terminal for data stream synchronization in multimodal services. The support capability of the terminal for data stream synchronization in multimodal services may include supporting data stream synchronization in multimodal services or not supporting data stream synchronization in multimodal services. By reporting the capability information, the terminal enables the network device to send resource configuration information to the terminals that support data stream synchronization in multimodal services in a targeted manner. For terminals that do not support data stream synchronization in multimodal services, the network device may use the original method for resource scheduling and data transmission. This can improve the overall compatibility.

[0029] In some possible implementation manners, the threshold may be configured by the network device or be a predefined value. Among them, the manner in which the network device configures the threshold can achieve real-time configuration and can adjust the threshold according to service requirements, so that better effects can be obtained when performing resource configuration.

[0030] In some possible implementation manners, the terminal may receive the configuration information sent by the network device, where the configuration information is used to indicate the data streams to be synchronized, and the data streams to be synchronized include a first data stream and a second data stream. This can monitor the delay of specific data streams configured by the network side to meet personalized service requirements.

[0031] In some possible implementation manners, this method may be executed by the terminal. The terminal is connected to multiple sensing devices, and the sensing device may be, for example, an XR device, such as an XR glasses or a handle. Each sensing device among the multiple sensing devices is used to collect at least one data stream of the multimodal service. This can implement resource configuration for multimodal services based on the same terminal and ensure the synchronization of data streams in multimodal services.

[0032] The second aspect of this application provides a data synchronization method, which may be executed by the network device. The network device may be an access network device, such as a base station, a wireless access point, a transmission and reception point, a wireless relay node, a wireless backhaul node, etc.

[0033] Specifically, the network device receives first information from the terminal, where the first information is used to indicate that the delay of the first data stream relative to the second data stream in the service is greater than or equal to the threshold, and then sends second information to the terminal, where the second information includes resource configuration information for the first data stream and / or the second data stream.

[0034] In this method, the network device may configure the resources for transmitting the first data stream and / or the second data stream according to the first information sent by the terminal, which is used to indicate the time delay timeout of the first data stream relative to the second data stream in the multi-modal service, so as to ensure the synchronization of data streams of different modalities, meet the service requirements, and improve the user experience.

[0035] In some possible implementation manners, the network device may also configure the resources for transmitting the first data stream and / or the second data stream, and obtain the resource configuration information for the first data stream and / or the second data stream. For example, the network device may configure the resources for transmitting the first data stream and / or the second data stream in terms of data stream granularity or multi-modal service granularity, so as to shorten the time delay of the first data stream relative to the second data stream and ensure the synchronization between the first data stream and the second data stream.

[0036] In some possible implementation manners, the first information includes:

[0037] The identifier of the first data stream and the identifier of the second data stream; or,

[0038] At least one of the time delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the identifier of the first data stream, the identifier of the second data stream; or,

[0039] The identifier of the first logical channel and the identifier of the second logical channel, where the first logical channel is used to transmit the first data stream and the second logical channel is used to transmit the second data stream; or,

[0040] At least one of the time delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the identifier of the first logical channel, the identifier of the second logical channel; or,

[0041] The identifier of the first data radio bearer (DRB) and the identifier of the second DRB, where the first DRB is used to transmit the first data stream and the second DRB is used to transmit the second data stream; or,

[0042] At least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the identifier of the first DRB, the identifier of the second DRB; or,

[0043] The mode of the first data stream and the mode of the second data stream; or,

[0044] At least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the mode of the first data stream, the mode of the second data stream; or,

[0045] An enumerated value, which is used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold; or,

[0046] At least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the enumerated value, which is used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold.

[0047] In some possible implementation manners, receiving first information from a terminal, including:

[0048] Receiving user equipment assistance information UAI from the terminal, where the UAI includes the first information; or,

[0049] Receiving a media access control layer control element MAC CE from the terminal, where the MAC CE includes the first information; or,

[0050] Receiving a packet data convergence protocol PDCP layer control message from the terminal, where the PDCP layer control message includes the first information.

[0051] In this method, the network device supports obtaining the first information in different forms such as user equipment assistance information, MAC CE, or PDCP layer control messages, which has high flexibility and improves the overall usability.

[0052] A third aspect of this application provides an electronic device, including: a memory and at least one processor. The memory is used to store a program, and the at least one processor is used to run the program so that the electronic device implements the data synchronization method provided in the first aspect of this application.

[0053] The fourth aspect of the present application provides an electronic device, including: a memory and at least one processor. The memory is used to store a program, and the at least one processor is used to run the program so that the electronic device implements the data synchronization method provided in the second aspect of the present application.

[0054] The fifth aspect of the present application provides a communication system, including a first device and a second device. The first device and the second device are used to execute the data synchronization method provided in the third aspect or the fourth aspect of the present application.

[0055] The sixth aspect of the present application is a computer storage medium, used to store a computer program, which when executed, is used to implement the data synchronization method provided in the first aspect or the second aspect of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a schematic diagram of a communication system architecture disclosed in an embodiment of the present application;

[0057] Figure 2 It is a schematic diagram of an application scenario disclosed in an embodiment of the present application;

[0058] Figure 3 It is a flowchart of a data synchronization method disclosed in an embodiment of the present application;

[0059] Figure 4 It is a schematic diagram for determining the time delay of different data streams disclosed in an embodiment of the present application;

[0060] Figure 5 It is a schematic diagram of the format of a medium access control layer control element disclosed in an embodiment of the present application;

[0061] Figure 6 It is a schematic diagram of the format of another medium access control layer control element disclosed in an embodiment of the present application;

[0062] Figure 7 It is a schematic diagram of the format of yet another medium access control layer control element disclosed in an embodiment of the present application;

[0063] Figure 8 It is a schematic diagram of the format of a packet data convergence protocol layer control message disclosed in an embodiment of the present application;

[0064] Figure 9 It is a flowchart of a data synchronization method disclosed in an embodiment of the present application;

[0065] Figure 10 It is a schematic diagram of an application scenario for implementing data synchronization by sending a first piece of information through a radio resource control message disclosed in an embodiment of the present application;

[0066] Figure 11 Schematic diagram of an application scenario for data synchronization by sending a first message through a media access control layer control element disclosed in an embodiment of the present application;

[0067] Figure 12 Schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application;

[0068] Figure 13 Schematic diagram of the structure of another electronic device disclosed in an embodiment of the present application. Detailed implementation manners

[0069] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and claims of the present application, the singular forms "a", "an", "the", "above", "the foregoing", "this" are also intended to include, for example, the expression "one or more", unless the context clearly indicates otherwise. It should also be understood that in the embodiments of the present application, "one or more" means one, two or more than two; "and / or" describes the association relationship of associated objects and means that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0070] Reference to "an embodiment" or "some embodiments" etc. described in this specification means that specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Thus, the statements "in an embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0071] The multiple related to the embodiments of the present application means greater than or equal to two. It should be noted that in the description of the embodiments of the present application, the terms "first", "second", etc. are only used for the purpose of distinguishing descriptions and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0072] The technical solution of the embodiment of the present application can be applied to various communication systems, such as: Global System for Mobile Communications (GSM) system, Code Division Multiple Access (CDMA) system, Wideband Code Division Multiple Access (WCDMA) system, General Packet Radio Service (GPRS), Long Term Evolution (LTE) system, LTE Frequency Division Duplex (FDD) system, LTE Time Division Duplex (TDD), Universal Mobile Telecommunication System (UMTS), Worldwide Interoperability for Microwave Access (WiMAX) communication system, 5th generation (5G) system or New Radio (NR), and future communication systems.

[0073] To address the challenges of wireless broadband technology and maintain the leading edge of the 3GPP network, the 3GPP standards group has developed the next-generation mobile communication network architecture (next generation system), known as the 5G network architecture. This architecture not only supports the access of wireless technologies defined by the 3GPP standards group (such as LTE, etc.) to the 5G core network (5G core network, 5GC), but also supports non-3GPP access technologies to access the 5GC through the non-3GPP interworking function (non-3GPP interworking function, N3IWF), the trusted non-3GPP gateway function (trusted non-3GPP gateway function, TNGF), the trusted WLAN interworking function (trusted WLAN interworking function, TWIF), or the next-generation access gateway (next generation packet data gateway, NG-PDG). Among them, the core network functions are divided into user plane function (user plane function, UPF) network elements and control plane function (control plane function, CPF) network elements. The UPF is mainly responsible for the forwarding of packet data, quality of service (quality of service, QoS) control, billing information statistics, etc. The CPF is mainly responsible for user registration authentication, mobility management, and sending packet forwarding policies, QoS control policies, etc. to the UPF, and can be further divided into the access and mobility management function (access and mobility management function, AMF) and the session management function (session management function, SMF).

[0074] Core network devices include, for example, a mobility management entity (mobility management entity, MME), a broadcast multicast service center (broadcast multicast service center, BMSC), etc., or may also include the corresponding functional entities in the 5G system, such as the core network control plane (control plane, CP) or user plane (user plan, UP) network functions, such as: SMF, AMF, etc. Among them, the core network control plane can also be understood as the core network control plane function (control plane function, CPF) entity.

[0075] Figure 1 This is an example of a communication system architecture applicable to the embodiments of this application. Among them, the functions of the user equipment and each network entity are as described below.

[0076] Terminal: It can be referred to as a terminal device, subscriber unit, terminal station, terminal agent, terminal device, access terminal, terminal in V2X communication, user unit, user equipment (UE), user station, mobile station, mobile station (MS), remote station, remote terminal, mobile device, user terminal, wireless communication device, user agent or user device.

[0077] The user equipment in the embodiments of the present application may also be a mobile phone, a tablet computer (pad), a computer with wireless transceiver function, a holographic projector, a video player, a virtual reality (VR) terminal, an augmented reality (AR) terminal, a wireless terminal in industrial control, a tactile terminal device, a vehicle-mounted terminal device, a wireless terminal in self-driving, a wireless terminal in remote medical, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with wireless communication function, a computing device or other processing devices connected to a wireless modem, a vehicle-mounted device, a wearable device, a terminal in a 5G network or a terminal in a future evolved network, etc.

[0078] Among them, wearable devices, also known as wearable intelligent devices, are the general term for devices developed by applying wearable technologies to the intelligent design of daily wear, including but not limited to XR glasses (such as AR glasses or VR glasses), gloves, watches, clothing, shoes, etc. Wearable devices are portable devices that are directly worn on the body or integrated into the user's clothes or accessories. Wearable devices are not just a hardware device, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable intelligent devices include those with complete functions and large sizes that can achieve complete or partial functions without relying on a smartphone, such as smart watches or smart glasses, etc., and those that focus on a certain type of application function and need to cooperate with other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring. It should be noted that wearable devices (such as XR devices like XR glasses) can also integrate the functions of a smartphone. For example, XR devices can also integrate a Subscriber Identity Module (SIM) or an Embedded-SIM (eSIM) for cellular communication.

[0079] Radio Access Network (RAN): A network composed of multiple 5G-RAN nodes that implements wireless physical layer functions, resource scheduling, wireless resource management, wireless access control, and mobility management functions. 5G-RAN is connected to the UPF through the user plane interface N3 for transmitting data of the terminal device; 5G-RAN establishes a control plane signaling connection with the AMF through the control plane interface N2 for functions such as wireless access bearer control. The RAN can be any device with wireless transceiver functions, including but not limited to 5G base stations (gNB), evolved base stations (eNB), wireless access points (WiFi AP), world interoperability for microwave access base stations (WiMAX BS), transmission receiving points (TRP), wireless relay nodes, wireless backhaul nodes, etc.

[0080] The access network device (i.e., the network device of the access network) in the embodiments of this application can also be a device for communicating with a terminal device. The access network device can be a base transceiver station (BTS) in a global system of mobile communication (GSM) system or a code division multiple access (CDMA) system, or a node B (NB) in a wideband code division multiple access (WCDMA) system, or an evolutional node base (eNB) in an LTE system, or a radio controller in a cloud radio access network (CRAN) scenario, or the access network device can be a relay station, an access point, a vehicle-mounted device, a wearable device, and an access network device in a future 5G network or an access network device in a future evolved PLMN network, etc. The embodiments of this application do not limit this.

[0081] In NR, the functions of the base station are divided into two parts, called the separation of the centralized unit (CU) - distributed unit (DU). From the perspective of the protocol stack, the CU includes the RRC layer and the PDCP layer of the LTE base station, and the DU includes the radio link control (RLC) layer, the media access control (MAC) layer, and the physical (PHY) layer of the LTE base station. In the deployment of an ordinary 5G base station, the CU and the DU can be physically connected by optical fibers, and there is a specifically defined F1 interface logically for communication between the CU and the DU. From the perspective of functions, the CU is mainly responsible for radio resource control and configuration, inter-cell mobility management, bearer management, etc. The DU is mainly responsible for scheduling, physical signal generation and transmission.

[0082] Among them, the above base station can be a macro base station, a micro base station, a pico base station, a small station, a relay station, a balloon station, etc.

[0083] SMF: The control plane function mainly responsible for the session management of terminal devices, including the selection and control of the user plane function (UPF), the allocation of Internet Protocol (IP) addresses, the QoS management of sessions, obtaining policy and charging control (PCC) policies (from the PCF), etc.

[0084] UPF: As the anchor point of the protocol data unit (PDU) session connection, it is responsible for filtering data packets of terminal devices, data transmission / forwarding, rate control, generating charging information, etc., and provides a connection to the data network (DN).

[0085] PCF: Provides configuration policy information for terminal devices and provides policy information for controlling terminal devices to network control plane network elements (such as SMF); generates access policies and QoS flow control policies for terminal devices.

[0086] AF: Interacts with network elements of the core network to provide some services. For example, it interacts with the PCF for service policy control, interacts with the NEF to obtain some network capability information or provide some application information to the network, and provides some data network access point information to the PCF to generate corresponding routing information for data services.

[0087] In the embodiments of this application, the terminal device is connected to the RAN device wirelessly, and the RAN network element is connected to the 5GC device wirelessly or wired. The 5GC device and the RAN network element can be independent different physical devices, or the functions of the 5GC device and the logical functions of the RAN network element can be integrated on the same physical device, or the functions of part of the 5GC device and part of the RAN network element can be integrated on a physical device. The terminal device can be fixed in position or movable.

[0088] The 5GC device mainly includes the above-mentioned PCF network element, SMF network element, UPF network element, etc.

[0089] It should be noted that the above-mentioned "network element" can also be referred to as an entity, device, apparatus, or module, etc., and this application does not particularly limit it. And in this application, for the convenience of understanding and description, the description of "network element" is omitted in some descriptions. For example, the NEF network element is abbreviated as NEF. In this case, the "NEF" should be understood as the NEF network element or NEF entity. Hereinafter, the description of the same or similar situations is omitted.

[0090] It should be noted that Figure 1The naming of each network element included is just a name, which does not limit the function of the network element itself. In the 5G network and other future networks, the above-mentioned network elements may also have other names, and the embodiments of the present application do not make specific limitations in this regard. For example, in the 6G network, some or all of the above-mentioned network elements may use the terms in 5G, or may have other names, etc. A unified description is given here and will not be elaborated below.

[0091] It should be noted that Figure 1 the various network elements in are not necessarily present at the same time, and which network elements are needed can be determined according to requirements. Figure 1 The connection relationship between the various network elements in is not uniquely determined either and can be adjusted according to requirements.

[0092] It can be understood that the above-mentioned network elements or functions can be either network elements in hardware devices, software functions running on dedicated hardware, or virtualized functions instantiated on a platform (such as a cloud platform).

[0093] Figure 2 is a schematic diagram of an application scenario applicable to the present application provided by the present application. As Figure 2 shown, the embodiments of the present application can be applied to a multi-modal service scenario. The multi-modal service scenario may include an AF network element and an XR device. Among them, the AF network element may be an XR server, and the XR device may be an XR head-mounted display (HMD) or XR glasses. Among them, the XR device may also be configured with a controller (such as a handle). The XR server can receive video frames, audio data, haptic data, or pose & control data from the XR device side, and return the processed video frames, audio data, haptic data, or pose & control data to the XR device. Among them, the XR server and the XR device interact various commands and feedback signals through the communication link in the network architecture to form a global control loop.

[0094] In the multi-modal service application scenario, multiple data streams (such as QoS flows) are required to transmit different data types such as video, audio, haptic, or pose & control. Among them, one QoS flow can be used to transmit one data type. In this example, the multi-modal service includes four service types. Therefore, 4 QoS flows can be used for data transmission.

[0095] There is a correlation between various data streams of multimodal services in terms of space and time. Spatial correlation refers to the correlation of different data streams at the same location in space, and temporal correlation refers to the correlation of different data streams over a period of time. The spatial correlation and temporal correlation are illustrated below with examples. Exemplarily, when a human touches objects with different surface textures and materials, the tactile sensations are also different. There is a certain correlation between the tactile data and the object surface images in the video data, and this correlation is also a specific form of spatial correlation. By utilizing the spatial correlation between different data streams, data reconstruction can be assisted. For example, when tactile data is transmitted over the network, due to channel fluctuations or network congestion, some data packets are lost. At this time, the spatial correlation between different data streams in multimodal services can be used to recover and reconstruct the damaged data. In this example, the video data can be used to recover the tactile data to ensure the user experience of multimodal services.

[0096] In other examples, when a person is speaking, the lip shape of the person in the video data can change. There is a certain correlation between the audio data in a period of time and the lip shape images in the video data before and after this period of time, and this correlation is also a specific form of temporal correlation. By utilizing the temporal correlation between different data streams, data reconstruction can be assisted. For example, when audio data is transmitted over the network and some data packets are lost due to channel fluctuations or network congestion, the temporal correlation between different data streams in multimodal services can be used to recover and reconstruct the damaged data. In this example, the video data can be used to recover the audio data to ensure the user experience of multimodal services.

[0097] In Figure 2 the example, the data collected by the XR device, such as video frames, audio data, tactile data, pose and control data, can be transmitted to the access network through a terminal (such as a UE), for example, transmitted to an access network device (such as a gNB). Among them, the terminal can map different types of data to corresponding bearers respectively, and the data transmitted on each bearer can form a QoS flow (QoS flow). The access network device can transmit the above data to the 5GC in the form of a QoS flow, and then the 5GC transmits the above data to the XR server. It should be noted that the XR device can also communicate directly with the access network device. For example, when the XR device integrates an eSIM or the function of the UE is integrated into the XR device, the XR device can directly connect to the access network device. Correspondingly, the XR device can directly transmit the collected data to the access network device, for example, directly transmit it to the access network device in the form of a QoS flow, without passing through the UE.

[0098] Correspondingly, the XR server can return a multi-modal data stream to the XR device. Specifically, the XR server can first return the multi-modal data stream to the 5GC, and the 5GC transmits the multi-modal data stream to the access network device in the form of a QoS flow. The access network device transmits the multi-modal data stream to the corresponding UE respectively, and then the UE transmits the received data stream to the XR device.

[0099] It should be noted that Figure 2 This is an example of a multi-modal scenario where the XR device is connected to the network through a single terminal, such as a multi-modal scenario based on the same UE or intra-UE. Multiple XR devices can be connected to the network through the same UE, and this UE can serve as a unified entry or exit for multiple XR devices to realize interaction with the XR server.

[0100] In addition to the application scenarios of the architecture between the above application server and terminal device, the application scenarios of this application can also be constructed by multiple terminal devices. In the scenario constructed by multiple terminal devices, one terminal device can be an XR device, and the other terminal device can be a remote robot. The XR device can control the operation of the remote robot. The construction of the application scenarios of this application is not limited to the above scenarios, and all application scenarios applicable to the technical solutions of this application are within the protection scope of this application.

[0101] It should be noted that when there is a multi-modal service requirement in the communication system, the SMF will establish a corresponding PDU session. Generally, one PDU session is established for one multi-modal service. One PDU session can correspond to multiple radio bearers (RB), and one PDU session can correspond to multiple data streams with different QoS requirements.

[0102] The synchronization of data streams greatly affects the user experience. For example, when the audio data stream and the video data stream are out of sync, the lip movement of the video picture may not correspond to the audio. Currently, the network side usually adopts an independent scheduling scheme for different data streams, resulting in a relatively large delay of one data stream relative to another in a multi-modal service scenario. For example, the video data stream arrives a long time after the audio data stream arrives, and it is difficult to synchronize different data streams.

[0103] In view of this, the present application provides a data synchronization method. Specifically, the terminal can obtain the latency of the first data stream (such as QoS flow1 for transmitting video data) relative to the second data stream (such as QoS flow2 for transmitting tactile data) in a multimodal service. When the latency of the first data stream relative to the second data stream is greater than or equal to a threshold, the terminal can send a first message to a network device (such as an access network device, including but not limited to a base station), and the first message can explicitly or implicitly indicate that the latency of the first data stream relative to the second data stream is greater than or equal to the threshold. Then, the terminal receives a second message sent by the network device, and the second message includes resource configuration information of the first data stream and / or the second data stream. The terminal can determine the resources for transmitting the first data stream and / or the resources for transmitting the second data stream according to the resource configuration information.

[0104] In this method, the terminal can monitor the latency of different data streams in a multimodal service. When the latency is greater than or equal to the threshold, it can notify the network device to configure the resources of the data stream to ensure the synchronization of different data streams, meet the service requirements, and improve the user experience.

[0105] To make the technical solution of the present application clearer and easier to understand, the data synchronization method of the embodiments of the present application will be introduced below with reference to the accompanying drawings.

[0106] See Figure 3 The flowchart of a data synchronization method shown, and the method includes:

[0107] S302. The terminal obtains the latency of the first data stream relative to the second data stream in a multimodal service. When the latency of the first data stream relative to the second data stream is greater than or equal to a threshold, execute S304. The threshold is a threshold corresponding to the service type of the first data stream and the service type of the second data stream.

[0108] A multimodal service can include multiple interrelated data streams, and the multiple data streams can come from different data sources. Each data stream can be regarded as a type (mode) of data stream associated with the same service, such as an audio data stream, a video data stream, a location data stream, or a tactile data stream. Each data stream in a multimodal service is also called a multimodal service flow. In some examples, the multimodal service can include but not be limited to XR services and cloud game services.

[0109] The first data stream and the second data stream can be different data streams in a multimodal service. Usually, the first data stream and the second data stream can be data streams of different modalities or different service types. For example, the first data stream can be a video data stream, and the second data stream can be a tactile data stream. Taking the example of a user playing cloud games through an XR device, the first data stream can be a video data stream formed by video data collected by the XR glasses, and the second data stream can be a tactile data stream formed by tactile data collected by the gamepad.

[0110] The latency of the first data stream relative to the second data stream can be the time difference between the first data stream and the second data stream. Packets in a data stream usually include timestamps. The timestamp reflects the sampling moment of the first byte of data in a Real-time Transport Protocol (RTP) packet. The initial value of the timestamp at the start of a session is also randomly selected. If two streams start sampling simultaneously, the RTP timestamp can be used for synchronization between the data streams. As Figure 4 shown, the starting time points of the two data streams are the same. According to the sampling period, the absolute time value of each frame can be determined, and then the time difference (i.e., latency) between the two data streams can be determined.

[0111] In specific implementation, the terminal can obtain the first packet in multiple buffers in the Packet Data Convergence Protocol (PDCP) layer. Among them, the buffer can be a queue, and the queue includes multiple elements, and each element can store a packet. Usually, the packets of a data stream can be stored in one buffer. The packets of different data streams can be stored in different buffers. In some examples, multiple data streams can also be stored in one buffer. Different data streams in the same buffer can be identified by port numbers. For the sake of convenience in description, the following takes the example that the packets of a data stream are stored in one buffer for illustration.

[0112] Among them, the PDCP layer belongs to the second layer of the radio interface protocol stack and is used to process Radio Resource Control (RRC) messages on the control plane and Internet Protocol (IP) packets on the user plane. The data packets in the buffer of the PDCP layer can be Service Data Units (SDUs). An SDU is also called a service data unit and is a data set for the user service of a specified layer. When transmitted to the receiving party, the data (i.e., the service part) of the same protocol layer does not change. When the data reaches the lower layer, the lower layer encapsulates the data in a protocol data unit (PDU) for transmission. The terminal can obtain the RTP header of the SDU, extract the timestamp from the RTP header, and determine the delay of different data streams according to the timestamps extracted from the RTP headers of the first SDUs in different buffers. For example, the terminal can determine the delay of the first data stream relative to the second data stream in the above manner. This delay can be the difference determined according to the timestamps extracted from the RTP headers of the first SDUs in different buffers, and is also called the synchronization time difference.

[0113] The terminal can compare the delay of the first data stream relative to the second data stream and the threshold corresponding to the service type of the first data stream and the service type of the second data stream (such as the threshold of the first service type relative to the second service type). Among them, the synchronization requirements for data streams of different service types can be different. Correspondingly, the thresholds corresponding to data streams of different service types (which can also be called synchronization delay thresholds or synchronization thresholds) can be different. This threshold can be configured by the network device or predefined through the protocol. Correspondingly, the threshold can be a value predefined in the protocol. In some possible implementation manners, the mapping relationship between the service type and the threshold can be predefined, and the terminal can store the mapping relationship between the service type and the synchronization delay threshold for subsequent comparison of the delays and thresholds of different data streams.

[0114] Among them, the terminal can store the mapping relationship between the service type and the threshold in a table as follows:

[0115] Table 1 Mapping relationship between service type and threshold

[0116]

[0117] In Table 1, "audio" represents audio and "tactile" represents touch. Taking "audio" and "tactile" as examples, "audiodelay" represents the maximum delay of audio relative to tactile, that is, after tactile arrives, audio should arrive no later than this maximum delay. If it arrives after this maximum delay, audio and tactile are out of sync. "Tactiledelay" represents the maximum delay of tactile relative to audio, that is, after audio arrives, tactile should arrive no later than this maximum delay. If it arrives after this maximum delay, tactile and audio are out of sync.

[0118] When the delay of the first data stream relative to the second data stream is greater than or equal to the threshold of the corresponding type, the operation of S304 can be triggered. Taking audio and touch as examples below, when the terminal obtains the delay of the data stream of audio type relative to the data stream of touch type, this delay can be compared with the threshold of audio relative to touch (such as 50 ms). If the delay of the data stream of audio type relative to the data stream of touch type is greater than or equal to the threshold of audio relative to touch, the operation of S304 can be triggered.

[0119] It should be noted that the terminal can also receive the configuration information sent by the network device. This configuration information can indicate the information of the data streams to be synchronized. Among them, the data streams to be synchronized can include the first data stream and the second data stream. The configuration information can include at least one of the identifier of the data stream to be synchronized, the identifier of the logical channel used by the data stream to be synchronized, the modality of the data stream to be synchronized, and the identifier of the data radio bearer of the data stream to be synchronized. Correspondingly, the terminal can obtain the delay of the data stream indicated by the configuration information according to the above configuration information. For example, the configuration information can include the identifier of the first data stream and the identifier of the second data stream, and the terminal can obtain the delay of the first data stream relative to the second data stream according to the configuration information. Further, the configuration information can also include the threshold corresponding to the service type of the data stream to be synchronized, or the threshold corresponding to the service type to be synchronized. In this way, the terminal can compare the delay of the data stream to be synchronized and the threshold corresponding to the data stream to be synchronized according to the configuration information.

[0120] It should also be noted that the multi-modal service in this embodiment can be a single-terminal multi-modal service. Among them, the terminal is connected to multiple sensing devices (such as XR devices or other sensing devices), and each sensing device in the multiple sensing devices is used to collect data streams of at least one modality. The terminal can obtain the delay of the first data stream relative to the second data stream in the above multi-modal service, and compare the delay and the threshold corresponding to the service type of the above data streams (such as the thresholds corresponding to the service type of the first data stream and the service type of the second data stream).

[0121] S304. The terminal sends the first information to the network device.

[0122] The first information can explicitly or implicitly indicate that the latency of the first data stream relative to the second data stream is greater than or equal to a threshold. Here, the first data stream and the second data stream can be data streams of different modalities in a multi-modal service. In some cases, the first data stream and the second data stream can be video streams of different modalities. For example, the first data stream can be a video stream and the second data stream can be an audio stream. In other cases, the first data stream and the second data stream can be video streams of the same modality. For example, both the first data stream and the second data stream can be video streams. The first information can indicate that the latency of the first data stream relative to the second data stream is greater than or equal to the threshold through different implementation manners. The following will respectively elaborate on different implementation manners in detail.

[0123] In the first implementation manner, the first information can include the identifier of the first data stream and the identifier of the second data stream. Here, the first data stream and the second data stream can be transmitted in the form of QoS flows. Correspondingly, the identifier of the first data stream and the identifier of the second data stream can be QoS flow identifiers (QFIs), such as QFI 1 and QFI 2.

[0124] In this implementation manner, the first information can be sent to the network device in the form of an array or a sequence.

[0125] For example, the first information can adopt the following format:

[0126] The first information {

[0127] data stream sequence {QFI 1, QFI 2}

[0128] }

[0129] Here, QFI 1 and QFI 2 are the identifiers of the data streams. It should be noted that in this example, sequence {QFI 1, QFI 2} indicates that the latency of data stream 1 relative to data stream 2 is greater than or equal to the threshold. It should be noted that the terminal and the network device can first negotiate the format of the sequence, or the format of the sequence can be agreed upon through a protocol. For example, the format of the sequence can be that the identifier of the later-arriving data stream is arranged in the front and the identifier of the earlier-arriving data stream is arranged in the back. In other possible implementation manners of the embodiments of the present application, sequence {QFI 1, QFI 2} can also indicate that the latency of data stream 2 relative to data stream 1 is greater than or equal to the threshold. In this case, if there is a latency of data stream 1 relative to data stream 2, it can be indicated by sequence {QFI 2, QFI1}.

[0130] In the second implementation, the first information may include the identifier of the first data stream, the identifier of the second data stream, and the latency of the first data stream relative to the second data stream. The latency can be determined by the system frame number (SFN), subframe, time slot, or absolute time, and the latency unit can be symbol, time slot, subframe, frame, or millisecond. For ease of description, the following uses the example where the latency is determined by absolute time and the latency unit is millisecond.

[0131] In this implementation, the first information can be sent to the network device in the form of an array or sequence. For example, the first information can be in the following format:

[0132] First information {

[0133] Data stream sequence {QFI 1, QFI 2}

[0134] Latency sequence {t delay}

[0135] }

[0136] where QFI 1 and QFI 2 are the identifiers of the data streams, and t delay is the latency (in milliseconds).

[0137] This example uses two arrays to provide the identifiers of the data streams and the latency respectively. In actual applications, the identifiers of the data streams and the latency can also be provided through one array, as shown below:

[0138] First information {

[0139] Data stream and latency sequence {QFI 1, QFI 2, t delay}

[0140] }

[0141] In the third implementation, the first information may include the modality of the first data stream and the modality of the second data stream. Among them, the modality can be identified by the modality name or modality type. For example, the modality of the first data stream can be video (or visual), and the modality of the second data stream can be tactile.

[0142] In this implementation, the first information can be sent to the network device in the form of an array or sequence. For example, the first information can be in the following format:

[0143] First information {

[0144] Modality sequence {modal type 1, modal type 2}

[0145] }

[0146] Among them, modal type 1 and modal type 2 are the modalities of the first data stream and the second data stream (specifically, the first modality and the second modality).

[0147] In the fourth implementation, the first information may include the modality of the first data stream, the modality of the second data stream, and the delay of the first data stream relative to the second data stream.

[0148] In this implementation, the first information may be sent to the network device in the form of an array or a sequence. For example, the first information may adopt the following format:

[0149] First information{

[0150] Modality sequence{modal type 1, modal type 2}

[0151] Delay sequence{t delay}

[0152] }

[0153] Among them, modal type 1 and modal type 2 are modalities, and t delay is the delay, for example, the delay of the first data stream relative to the second data stream.

[0154] This example uses two arrays to provide the modality and the delay respectively. In actual applications, the modality and the delay may also be provided through one array, as shown below:

[0155] First information{

[0156] Modality and delay sequence{modal type 1, modal type 2, t delay}

[0157] }

[0158] In the fifth implementation, the first information may include the identifier of the first logical channel and the identifier of the second logical channel. Among them, the first logical channel is used to transmit the first data stream, and the second logical channel is used to transmit the second data stream.

[0159] In this implementation, the first information may be sent to the network device in the form of an array or a sequence. For example, the first information may adopt the following format:

[0160] First information{

[0161] Logical channel sequence{LCID 1, LCID 2}

[0162] }

[0163] Among them, LCID 1 and LCID 2 are the identifiers (identifiers, IDs) of the logical channel (logic channel). It should be noted that in this example, sequence{LCID 1, LCID 2} means that the delay of data stream 1 in logical channel 1 relative to data stream 2 in logical channel 2 is greater than or equal to the threshold. In other possible implementation manners of the embodiments of the present application, sequence{LCID 1, LCID 2} may also mean that the delay of data stream 2 in logical channel 2 relative to data stream 1 in logical channel 1 is greater than or equal to the threshold. In this case, if there is a delay of data stream 1 relative to data stream 2, it can be indicated by sequence{LCID 2, LCID 1}.

[0164] In the sixth implementation manner, the first information may include the identifier of the first logical channel, the identifier of the second logical channel, and the delay of the first data stream relative to the second data stream.

[0165] In this implementation manner, the first information may be sent to the network device through an array or a sequence. For example, the first information may adopt the following format:

[0166] First information{

[0167] Logical channel sequence{LCID 1, LCID 2}

[0168] Delay sequence{t delay}

[0169] }

[0170] Among them, LCID 1, LCID 2 are the identifiers of the logical channel, and t delay is the delay (in milliseconds).

[0171] This example uses two arrays to provide the identifiers of the logical channel and the delay respectively. In actual applications, the identifiers of the logical channel and the delay can also be provided through one array, as shown below:

[0172] First information{

[0173] Logical channel and delay sequence{LCID 1, LCID 2, t delay}

[0174] }

[0175] In the seventh implementation manner, the first information may include the identifier of the first data radio bearer (DRB) and the identifier of the second DRB. Among them, the first DRB is used to transmit the first data stream, and the second DRB is used to transmit the second data stream.

[0176] In this implementation manner, the first information may be sent to the network device in the form of an array or a sequence. For example, the first information may adopt the following format:

[0177] First information {

[0178] Data radio bearer sequence{DRB_ID 1, DRB_ID 2}

[0179] }

[0180] Among them, DRB_ID 1 and DRB_ID 2 are the identifiers of the DRB. It should be noted that in this example, sequence{DRB_ID 1, DRB_ID 2} indicates that the delay of data stream 1 carried by DRB_ID 1 relative to data stream 2 carried by DRB_ID 2 is greater than or equal to the threshold. In other possible implementation manners of the embodiments of the present application, sequence{DRB_ID 1, DRB_ID 2} may also indicate that the delay of data stream 2 carried by DRB_ID 2 relative to data stream 1 carried by DRB_ID 1 is greater than or equal to the threshold. In this case, if there is a delay between data stream 1 and data stream 2, it can be indicated by sequence{DRB_ID 2, DRB_ID 1}.

[0181] In the eighth implementation manner, the first information may include the identifier of the first DRB, the identifier of the second DRB, and the delay of the first data stream relative to the second data stream.

[0182] In this implementation manner, the first information may be sent to the network device in the form of an array or a sequence. For example, the first information may adopt the following format:

[0183] First information {

[0184] Data radio bearer sequence{DRB_ID 1, DRB_ID 2}

[0185] Delay sequence{t delay}

[0186] }

[0187] This example uses two arrays to provide the identifiers of the DRB and the delay respectively. In actual applications, the identifiers of the DRB and the delay can also be provided by one array, as shown below:

[0188] First information {

[0189] Data radio bearer and delay sequence {DRB_ID 1, DRB_ID 2, t delay}

[0190] }

[0191] In the ninth implementation, the first information includes an enumerated value that is used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold. Specifically, an enumeration is a user-defined type that consists of multiple string constants separated by commas. These constants are called enumerated values.

[0192] In this application, the user can define string constants 1, string constants 2... string constants n, which respectively indicate that the delay of a data stream of one modality relative to a data stream of another modality is greater than or equal to a threshold. For example, the user can define warning1, warning2, warning3, warning4, warning5, warning6 to respectively represent that the delay between data streams of three modalities is greater than or equal to a threshold. Among them, warning1 indicates that the delay of the data stream of modality 1 relative to the data stream of modality 2 is greater than or equal to a threshold, warning2 indicates that the delay of the data stream of modality 1 relative to the data stream of modality 3 is greater than or equal to a threshold, warning3 indicates that the delay of the data stream of modality 2 relative to the data stream of modality 1 is greater than or equal to a threshold, warning4 indicates that the delay of the data stream of modality 2 relative to the data stream of modality 3 is greater than or equal to a threshold, warning5 indicates that the delay of the data stream of modality 3 relative to the data stream of modality 1 is greater than or equal to a threshold, and warning6 indicates that the delay of the data stream of modality 3 relative to the data stream of modality 2 is greater than or equal to a threshold.

[0193] This example can provide the enumerated values using an array, as shown below:

[0194] First information {

[0195] Enumerated value sequence {warning1, warning6}

[0196] }

[0197] In the tenth implementation, the first information includes an enumerated value and the delay of the first data stream relative to the second data stream, where the enumerated value is used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold.

[0198] This example can provide the enumerated values and the delay using an array, as shown below:

[0199] First information {

[0200] Enumeration values sequence {warning1, warning6}

[0201] Time delay sequence {t1 delay , t2 delay}

[0202] }

[0203] Among them, warning1 and warning6 are enumeration values, and t1 delay , t2 delay are time delays. The above enumeration values and time delays have a corresponding relationship. In this example, warning1 corresponds to t1 delay , indicating that the time delay of the data stream in mode 1 relative to the data stream in mode 2 is t1 delay , and warning6 corresponds to t2 delay , indicating that the time delay of the data stream in mode 3 relative to the data stream in mode 2 is t2 delay .

[0204] This example uses two arrays to provide enumeration values and time delays respectively. In actual applications, it is also possible to provide enumeration values and time delays through one array, as shown below:

[0205] First information {

[0206] Enumeration value and time delay sequence {warning1, warning6, t1 delay , t2 delay}

[0207] }

[0208] Among them, the first half of the sequence represents the enumeration value, and the second half of the sequence represents the time delay corresponding to each enumeration value. In actual applications, it is also possible to set the time delay corresponding to each enumeration value after each enumeration value, as shown below:

[0209] First information {

[0210] Enumeration value and time delay sequence {warning1, t1 delay , warning6, t2 delay}

[0211] }

[0212] It should be noted that the above implementation manners are merely some exemplary implementation manners of the present application. In other possible implementation manners of the embodiments of the present application, the time delay in the first information may also be replaced by at least one of the time delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, and the rate difference between the first data stream and the second data stream. Among them, the buffer difference between the first data stream and the second data stream may be the difference between the buffer size of the first data stream and the buffer size of the second data stream, and the rate difference between the first data stream and the second data stream may be the difference between the rate of the first data stream and the rate of the second data stream. By providing the above information, richer information can be provided for the network side to configure resources for transmitting the first data stream or the second data stream, thereby obtaining better effects.

[0213] The above first information may be encapsulated in a message or other information and structure for transmission. For example, a terminal may send user equipment assistant information (UAI) to a network device, and the UAI includes the above first information. Another example is that the terminal may send a medium access control layer control element (MAC CE) to the network device, and the MAC CE includes the first information. Or, the terminal may send a PDCP layer control message, such as a PDCP layer control PDU, and the PDCP layer control message includes the first information.

[0214] Specifically, the UAI includes a traffic information structure (denoted as trafficInfo), and the traffic information structure includes synchronization information for multi-modal services (denoted as syncInfo). Among them, the synchronization information is an array including the identifier of the first data stream (such as QFI 1) and the identifier of the second data stream (such as QFI 2). Further, the array may also include the time delay of the first data stream relative to the second data stream.

[0215] It should be noted that when the syncInfo is represented by an array, it may include information on multiple groups of data streams. The following is illustrated with an example as follows:

[0216] PDU-SessionUL-TrafficInfo::=SEQUENCE{

[0217] pdu-SessionID PDU-SessionID,

[0218] qos-FlowUL-TrafficInfoList SEQUENCE(SIZE(1..maxNrofQFIs)) OF QOS-FlowUL-TrafficInfo

[0219] qos-FlowUL-TrafficSyncInfoList SEQUENCE(SIZE(1..maxNrofSync)) OF QOS-FlowUL-TrafficSyncInfo OPTIONAL,

[0220] }

[0221] QOS-FlowUL-TrafficSyncInfo ::= SEQUENCE {

[0222] qfi1 INTEGER(0..maxQFI),

[0223] qfi2 INTEGER(0..maxQFI),

[0224] delay INTEGER(0..1023) OPTIONAL,

[0225] }

[0226] Among them, qfi1 and qfi2 are the identifiers of the first data flow, and delay is the time delay of the first data flow relative to the second data flow. It should be noted that for other fields not explained or defined, reference can be made to the relevant content described in TS38.331, which will not be elaborated here.

[0227] When the first information is sent in the form of MAC CE, the MAC CE may include the identifier of the first logical channel (denoted as LCID 1) and the identifier of the second logical channel (denoted as LCID 2). Among them, LCID1 and LCID2 can be 6 bits.

[0228] Figure 5 、 Figure 6 、 Figure 7 respectively show several formats of the MAC CE. As Figure 5 shown, when LCID is greater than 34, the MAC CE carries LCID 1 and LCID 2 through 2 bytes. Among them, LCID 1 and LCID2 together occupy 12 bits, and the remaining 4 bits can be used as reserved bits. As Figure 6 shown, when LCID is equal to 33, 2 additional bytes are added to the MAC CE for filling the extended LCID (extendLCID, eLCID). As Figure 7As shown, when the LCID is equal to 34, an additional 1 byte is added to the MAC CE for filling the eLCID.

[0229] Furthermore, the MAC CE can also continue to be filled with a delay. For example, an additional byte can be added to fill the delay. The unit of the delay can be milliseconds, and the maximum length is 255 ms or 256 ms. Among them, when the delay starts counting from 0, the maximum length can be 255 ms, and when the delay starts counting from 1, the maximum length can be 256 ms.

[0230] When using the PDCP layer control message to send the first information, the enumerated values of the type field can also be increased. The increased enumerated values are used to indicate synchronization information. As Figure 8 shown, the PDCP layer control message includes a type field and an identification field. The value of the type field is the enumerated value indicating synchronization information. In this example, the value of the PDU Type field can be 011, and the identification field is filled with the identification of the first DRB (DRB_ID 1) and the identification of the second DRB (DRB_ID 2).

[0231] Similar to the MAC CE, the PDCP layer control message can also continue to be filled with a delay. For example, an additional byte can be added to fill the delay. The unit of the delay can be milliseconds, and the maximum length is 255 ms.

[0232] S306. The terminal receives the second information sent by the network device.

[0233] The second information may include resource configuration information for the first data stream and / or the second data stream. Among them, the terminal can receive the resource configuration information sent by the network device, or not receive the resource configuration information sent by the network device. Whether the terminal receives the resource configuration information sent by the network device is related to whether the network device performs resource configuration. When the network device receives the first information, it can configure the resources for transmitting the first data stream and / or the second data stream, or not configure the resources for transmitting the first data stream and / or the second data stream. For example, in the case where the multi-modal service has a relatively low requirement for synchronization, the resources for transmitting the first data stream and / or the second data stream can also not be configured. Among them, the purpose of configuring the resources for transmitting the first data stream and / or the second data stream is to synchronize the first data stream and the second data stream. Whether the first data stream and the second data stream are synchronized can also be affected by other factors. Therefore, configuring the resources for transmitting the data stream may achieve synchronization or may not achieve synchronization. It should be noted that configuring the resources for transmitting the data stream can also be for other purposes, and this application does not limit this.

[0234] The following takes the network device configuring the resources for transmitting the first data stream and / or the second data stream and the terminal receiving the resource configuration information sent by the network device as an example for illustration.

[0235] In some possible implementations, the network device can be configured at the granularity of data streams (such as QoS streams). Correspondingly, the resource configuration information can be resource configuration information at the data stream granularity. Specifically, the network device can configure the priority of the logical channel for transmitting the first data stream, the priority bit rate (PBR), or the bucket depth (Bucket Size Duration, BSD) of the token bucket according to the first information. Correspondingly, the resource configuration information can include the priority, PBR, or BSD of the logical channel (such as the first logical channel) for transmitting the first data stream. Among them, the resource configuration information can also include the identifier of the first logical channel. Alternatively, the network device can configure the priority, PBR, or BSD of the logical channel for transmitting the second data stream according to the first information. Correspondingly, the resource configuration information can include the priority, PBR, or BSD of the logical channel for transmitting the second data stream. Among them, the resource configuration information can also include the identifier of the second logical channel.

[0236] For ease of understanding, this application provides an example for illustration. In this example, the priority of the first logical channel is 3, and the priority of the second logical channel is 2. Among them, the smaller the value of the priority, the higher the priority. When the first information is obtained, the first information is used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to the threshold, the network device can configure the priority of the first logical channel to 1, or configure the priority of the second logical channel to 4.

[0237] Furthermore, the network device can also configure the priority, PBR, or BSD of the first logical channel and configure the priority, PBR, or BSD of the second logical channel according to the first information. For example, the first information can not only indicate that the delay of the first data stream relative to the second data stream is greater than or equal to the threshold, but also indicate the delay of the first data stream relative to the second data stream. If the delay is much greater than the threshold, the network device can reconfigure both the resources for transmitting the first data stream and the resources for transmitting the second data stream. For example, configure the priority of the first logical channel to a high priority and configure the priority of the second logical channel to a low priority.

[0238] In some other possible implementation manners, the network device may also be configured based on multi-modal services. Correspondingly, the resource configuration information may also be the resource configuration information for multi-modal service granularity. Specifically, the network device may reconfigure the uplink grant (UL grant) of the multi-modal service according to the first information. The uplink grant is also referred to as uplink scheduling permission or uplink scheduling authorization, which is a physical control information from the access network device, indicating that the terminal can transmit data through the resources specified by the UL grant after receiving the UL grant. When reconfiguring the uplink UL grant, the resources specified by the uplink grant may be increased. Correspondingly, the second information may be the UL grant.

[0239] Similar to the first information, the second information may be encapsulated in a message or other information or structure for transmission. For example, the terminal may receive an RRC message sent by the network device that includes the second information. Among them, the RRC message including the second information may include a resource reconfiguration message. In some examples, the resource reconfiguration message may include logical channel configuration information, denoted as RRCReconfig(LogicChannelConfig). In some examples, the logical channel configuration information may include logical channel parameters, such as the priority of the logical channel.

[0240] S308. The terminal determines the resources for transmitting the first data stream and / or the second data stream according to the second information.

[0241] Specifically, the second information may include the priority of the first logical channel and / or the priority of the second logical channel. The terminal may allocate resources for the data streams from the resources specified by the UL grant in the order of the priorities. The following is an example for illustration. The priority of the first logical channel is 3, and the priority of the second logical channel is 2. When the terminal receives the second information and the priority of the first logical channel in the second information changes to 1, the terminal may preferentially allocate resources for the first data stream from the resources specified by the UL grant. In this way, the terminal can determine the resources for transmitting the first data stream and / or the second data stream.

[0242] When the second information is the resource configuration information for multi-modal service granularity, such as the UL grant, the terminal may determine the resources of the multi-modal service according to the UL grant. The terminal may allocate resources for multiple data streams from the resources specified by the UL grant according to the priorities of the logical channels used for transmitting multiple data streams in the multi-modal service. In this way, the terminal can determine the resources for transmitting the first data stream and / or the second data stream.

[0243] Considering that the capabilities of different terminals may vary, the terminal can also send capability information to the network device, and the capability information can be the capabilities related to synchronization. The capability information can indicate the terminal's support for data stream synchronization in multimodal services. The terminal's support for data stream synchronization in multimodal services can include supporting data stream synchronization in multimodal services or not supporting data stream synchronization in multimodal services. Correspondingly, the network device can send a second piece of information to the terminal that supports data stream synchronization in multimodal services to instruct the terminal to determine the resources for transmitting the first data stream and / or the second data stream according to the second piece of information. For terminals that do not support data stream synchronization in multimodal services, the network device can use the original method for resource scheduling and data transmission. This can improve the overall compatibility.

[0244] Based on the above description, this application provides a data synchronization method. In this method, the terminal can monitor the latency of multiple data streams in a multimodal service, for example, the latency of one data stream relative to another data stream. When the latency is greater than or equal to a threshold, the network device can be notified to configure the resources of the data stream to ensure the synchronization of different data streams, meet the service requirements, and improve the user experience.

[0245] The above introduced the data synchronization method of this application from the perspective of the terminal. Next, the synchronization method of this application will be introduced from the perspective of the network device.

[0246] See Figure 9 The flowchart of a data synchronization method shown in the figure. This method includes the following steps:

[0247] S902. The network device receives the first piece of information from the terminal.

[0248] The network device can be an access network device, such as a base station. The first piece of information is used to indicate that the latency of the first data stream relative to the second data stream in the multimodal service is greater than or equal to a threshold. The threshold can be a threshold corresponding to the service type of the first data stream and the service type of the second data stream, for example, the threshold of the first service type relative to the second service type. Among them, the synchronization requirements of data streams of different service types can be different. Correspondingly, the thresholds (which can also be called synchronization latency thresholds, synchronization thresholds) corresponding to data streams of different service types can be different. The threshold can be configured by the network device or predefined through a protocol. Correspondingly, the threshold can be a predefined value.

[0249] Among them, the first piece of information can include multiple implementation manners. The following will be described separately.

[0250] In some possible implementation manners, the first piece of information can include the identifier of the first data stream and the identifier of the second data stream. Further, the first piece of information can also include the latency of the first data stream relative to the second data stream.

[0251] In some possible implementations, the first information may also include the identifier of the first logical channel and the identifier of the second logical channel. Among them, the first logical channel is used to transmit the first data stream, and the second logical channel is used to transmit the second data stream. Further, the first information may also include the latency of the first data stream relative to the second data stream.

[0252] In some possible implementations, the first information may also include the identifier of the first DRB and the identifier of the second DRB. Among them, the first DRB is used to transmit the first data stream, and the second DRB is used to transmit the second data stream. Further, the first information may also include the latency of the first data stream relative to the second data stream.

[0253] In some possible implementations, the first information may also include the modality of the first data stream and the modality of the second data stream. Further, the first information may also include the latency of the first data stream relative to the second data stream.

[0254] In some possible implementations, the first information may also include an enumerated value that is used to indicate that the latency of the first data stream relative to the second data stream is greater than or equal to a threshold. Further, the first information may also include the latency of the first data stream relative to the second data stream.

[0255] The above implementations are merely some exemplary implementations of the present application. In other possible implementations of the embodiments of the present application, the latency in the first information may also be replaced by at least one of the latency of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, and the rate difference between the first data stream and the second data stream. Among them, the buffer difference between the first data stream and the second data stream may be the difference between the buffer size of the first data stream and the buffer size of the second data stream, and the rate difference between the first data stream and the second data stream may be the difference between the rate of the first data stream and the rate of the second data stream. When the network side obtains the above information, it can more accurately configure the resources for transmitting the first data stream and / or the second data stream.

[0256] In specific implementation, the network device may obtain the above first information through different methods. Explanations are given separately below.

[0257] For example, a network device may obtain (e.g., receive from a terminal) user equipment assistance information (UAI), and the UAI includes first information. Accordingly, the network device may parse the UAI to obtain the first information. For example, the UAI may carry, via trafficInfo, the identifier of a first data stream and the identifier of a second data stream. Further, the UAI may also carry, via trafficInfo, a time delay. Accordingly, the network device may parse the UAI, e.g., parse the trafficInfo of the UAI, to obtain the identifier of the first data stream and the identifier of the second data stream. If the trafficInfo also carries the time delay of the first data stream relative to the second data stream, the above time delay may also be obtained by parsing the UAI.

[0258] Also for example, a network device may obtain (e.g., receive from a terminal) a MAC CE, and the MAC CE includes first information. Accordingly, the network device may parse the MAC CE to obtain the first information. For example, the MAC CE may include the identifier of a first logical channel and the identifier of a second logical channel. The network device may parse the MAC CE and read the content of the corresponding field to obtain the identifier of the first logical channel and the identifier of the second logical channel. If the MAC CE also includes the time delay of the first data stream relative to the second data stream, the above time delay may also be obtained by parsing the MAC CE.

[0259] Also for example, a network device may obtain (e.g., receive from a terminal) a PDCP layer control message, and the PDCP layer control message includes first information. Accordingly, the network device may parse the above PDCP layer control message to obtain the first information. For example, a PDCP control PDU may include the identifier of a first DRB and the identifier of a second DRB. The network device may parse the PDCP control PDU and read the content of the corresponding field to obtain the identifier of the first DRB and the identifier of the second DRB. If the PDCP control PDU also includes the time delay of the first data stream relative to the second data stream, the above time delay may also be obtained by parsing the PDCP control PDU.

[0260] S904: The network device sends second information to the terminal.

[0261] The second information includes resource configuration information for the first data stream and / or the second data stream. When the network device receives the first information, it may configure the resources for transmitting the first data stream and / or the second data stream, obtain the resource configuration information for the first data stream and / or the second data stream, or not configure the resources for transmitting the first data stream and / or the second data stream. For example, in the case where the synchronization requirement of the multi-modal service is relatively low, the resources for transmitting the first data stream and / or the second data stream may not be configured either. Among them, the purpose of configuring the resources for transmitting the first data stream and / or the second data stream is to synchronize the first data stream and the second data stream. Whether the first data stream and the second data stream are synchronized may also be affected by other factors. Therefore, configuring the resources for transmitting the data stream may or may not achieve synchronization. It should be noted that configuring the resources for transmitting the data stream may also have other purposes, which are not limited in this application.

[0262] In some possible implementation manners, the network device may configure by data stream granularity. Correspondingly, the resource configuration information may be the resource configuration information of data stream granularity. Specifically, the network device may configure the priority of the logical channel, PBR, or the bucket depth BSD of the token bucket for transmitting the first data stream according to the first information. Correspondingly, the resource configuration information may include the priority, PBR, or BSD for transmitting the first logical channel. Among them, the resource configuration information may further include the identifier of the first logical channel. Or, the network device may configure the priority of the logical channel, PBR, or BSD for transmitting the second data stream according to the first information. Correspondingly, the resource configuration information may include the priority, PBR, or BSD for transmitting the second data stream. Among them, the resource configuration information may further include the identifier of the second logical channel.

[0263] In some other possible implementation manners, the network device may also configure by multi-modal service granularity. Correspondingly, the resource configuration information may also be the resource configuration information of multi-modal service granularity. Specifically, the network device may reconfigure the UL grant of the multi-modal service according to the first information. When reconfiguring the uplink UL grant, the resources specified by the uplink grant may be increased. Correspondingly, the second information may be the UL grant.

[0264] When sending the second information, the network device may encapsulate the above second information in a message, other information, or a structure for transmission. Specifically, the network device may encapsulate the second information in an RRC message. This RRC message may be a resource reconfiguration message. In some examples, the resource reconfiguration message may include logical channel configuration information, denoted as RRCReconfig(LogicChannelConfig). Among them, the logical channel configuration information may include logical channel parameters, such as the priority of the logical channel.

[0265] Based on the above description, the present application provides a data synchronization method. In this method, the network device can configure the resources for transmitting the first data stream and / or the second data stream according to the first information sent by the terminal indicating the time delay timeout of the first data stream relative to the second data stream in the multi-modal service, so as to ensure the synchronization of data streams in different modalities, meet the service requirements, and improve the user experience.

[0266] Next, in combination with specific application scenarios, the data synchronization method of the present application will be introduced from the perspective of the interaction between the terminal and the network device.

[0267] See Figure 10 The application scenario schematic diagram of a data synchronization method shown below includes the following specific steps:

[0268] S1002. The terminal obtains the time delay of the first data stream relative to the second data stream in the multi-modal service. When the time delay is greater than or equal to the threshold, execute S1004.

[0269] The first data stream and the second data stream can be data streams in different modalities in the multi-modal service. For ease of understanding, in the embodiments of the present application, the first data stream is taken as an example of a data stream of the video type, and the second data stream is taken as an example of a data stream of the pose and control type. The terminal can obtain the first SDU (or PDU) in the PDCP layer video buffer and the first SDU (or PDU) in the pose and control buffer, extract the timestamp from the RTP header of the SDU, and determine the time delay of the first data stream relative to the second data stream according to the timestamps extracted from the RTP headers of the first SDUs in different buffers. When the time delay is greater than or equal to the threshold corresponding to the corresponding service type, the operation of executing S1004 can be triggered.

[0270] S1004. The terminal sends the user equipment assistance information UAI including the first information to the network device.

[0271] Specifically, the terminal can send the first information to the network device through the UAI architecture at the RRC layer. The first information is used to indicate that the time delay of the first data stream relative to the second data stream is greater than or equal to the threshold. Further, in addition to including the identifier of the first data stream and the identifier of the second data stream, the UAI may further include at least one of the time delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, and the rate difference between the first data stream and the second data stream.

[0272] S1006. The network device configures the resources for transmitting the first data stream and / or the second data stream according to the first information and obtains the second information.

[0273] The second information includes resource configuration information for the first data stream and / or the second data stream. Specifically, the network device may, based on the first information, configure resources for transmitting the first data stream and / or the second data stream at the granularity of a data stream or at the granularity of a multi-modal service, to obtain the resource configuration information. The specific configuration process may refer to Figure 9 the relevant content description in the embodiments, which will not be elaborated here.

[0274] S1008. The network device sends the second information to the terminal.

[0275] The network device may encapsulate the above second information in a message or other information or structure for transmission. Specifically, the network device may encapsulate the second information in an RRC message. This RRC message may be a resource reconfiguration message. In some examples, the resource reconfiguration message may include logical channel configuration information, denoted as RRCReconfig(LogicChannelConfig). Among them, the logical channel configuration information may include logical channel parameters, such as the priority of the logical channel.

[0276] S1010. The terminal determines resources for transmitting the data stream of the first mode and / or the data stream of the second mode according to the second information.

[0277] In some possible implementation manners, the second information may include the priority of the first logical channel and / or the priority of the second logical channel, and the terminal may allocate resources for the data stream from the resources specified by the UL grant in the order of the priorities.

[0278] In some other possible implementation manners, the second information is resource configuration information at the granularity of a multi-modal service. For example, when there is a UL grant, the terminal may determine the resources of the multi-modal service according to the UL grant, and then the terminal may allocate resources for multiple data streams from the resources specified by the UL grant according to the priorities of the logical channels for transmitting multiple data streams in the multi-modal service. In this way, the terminal may determine the resources for transmitting the first data stream and / or the second data stream.

[0279] Figure 10 Taking the example that the terminal sends the first information through an RRC message to achieve data synchronization for illustration, the following takes the example that the terminal sends the first information through a MAC CE to achieve data synchronization for illustration.

[0280] See Figure 11 the schematic diagram of the application scenario of a data synchronization method shown in the figure, which specifically includes the following steps:

[0281] S1102. The terminal obtains the delay of the first data stream relative to the second data stream in the multi-modal service. When the delay is greater than or equal to the threshold, execute S1104.

[0282] The first data stream and the second data stream may be data streams of different modalities in a multimodal service. For the specific implementation of the terminal to obtain the latency of the first data stream relative to the second data stream, reference may be made to the relevant description in S1002, which will not be elaborated here.

[0283] S1104. The terminal sends a MAC CE including first indication information to the network device.

[0284] Specifically, the terminal may, at the MAC layer, send first information to the network device through the MAC CE. The first information includes the identifier of the first logical channel and the identifier of the second logical channel, and is used to indicate that the latency of the first data stream relative to the second data stream is greater than or equal to a threshold. Further, in addition to including the identifier of the first logical channel and the identifier of the second logical channel, the MAC CE may further include at least one of the latency of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, and the rate difference between the first data stream and the second data stream.

[0285] S1106. The network device configures resources for transmitting the first data stream and / or the second data stream according to the first information.

[0286] S1108. The network device sends second information to the terminal.

[0287] S1110. The terminal determines the resources for transmitting the first data stream and / or the second data stream according to the second information.

[0288] Figure 11 The main difference between the illustrated embodiment and Figure 10 the illustrated embodiment lies in S1104. For the specific implementation of other steps, reference may be made to Figure 10 the relevant description in the illustrated embodiment, for example, the specific implementation of S1106 to S1110 may refer to Figure 10 the relevant description of S1006 to S1010 in the embodiment.

[0289] It should be noted that the relevant content of each embodiment of this application may be related and referenced to each other.

[0290] Figure 12 This is a composition example of an electronic device provided by an embodiment of this application. The electronic device may be a first device, including but not limited to a base station and a core network unit. Figure 12A simplified schematic diagram of a base station structure is shown. The base station includes a 1210 part, a 1220 part, and a 1230 part. The 1210 part is mainly used for baseband processing and controlling the base station, etc.; the 1210 part is usually the control center of the base station and can usually be called a processor, which is used to control the base station to execute the processing operations on the first device side in the above method embodiments. The 1220 part is mainly used for storing computer program codes and data. The 1230 part is mainly used for the transceiver of radio frequency signals and the conversion between radio frequency signals and baseband signals; the 1230 part can usually be called a transceiver module, a transceiver, a transceiver circuit, or a transceiver, etc. The transceiver module of the 1230 part can also be called a transceiver or a transceiver, etc., and it includes an antenna 1233 and a radio frequency circuit (not shown in the figure), where the radio frequency circuit is mainly used for radio frequency processing. Optionally, the devices used to implement the receiving function in the 1230 part can be regarded as a receiver, and the devices used to implement the transmitting function can be regarded as a transmitter, that is, the 1230 part includes a receiver 1232 and a transmitter 1231. The receiver can also be called a receiving module, a receiver, or a receiving circuit, etc., and the transmitter can be called a transmitting module, a transmitter, or a transmitting circuit, etc.

[0291] The 1210 part and the 1220 part may include one or more single boards, and each single board may include one or more processors and one or more memories. The processor is used to read and execute the programs in the memory to implement the baseband processing function and the control of the base station. If there are multiple single boards, the single boards can be interconnected to enhance the processing ability. As an alternative implementation, it can also be that multiple single boards share one or more processors, or multiple single boards share one or more memories, or multiple single boards share one or more processors at the same time.

[0292] For example, in one implementation, the transceiver module of the 1230 part is used to execute the transceiver-related processes performed by the base station (the first device) in the above method embodiments. The processor of the 1210 part is used to execute the processing-related processes performed by the base station in the above method embodiments.

[0293] It should be understood that Figure 12 only for example and not limitation, the above network device including a processor, a memory, and a transceiver may not depend on Figure 12 the structure shown.

[0294] Figure 13This is another example of the composition of the electronic device provided by the embodiment of the present application. The electronic device may be a second device, and the second device may be a terminal, including but not limited to electronic devices such as mobile phones and smart wearable devices (such as smart watches). Taking a mobile phone as an example, the electronic device may include a processor 310, an external memory interface 320, an internal memory 321, a display screen 330, a camera 340, antenna 1, antenna 2, a mobile communication module 350, a wireless communication module 360, etc.

[0295] It can be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0296] The processor 310 may include one or more processing units. For example, the processor 310 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0297] It can be understood that the interface connection relationship between the modules illustrated in this embodiment is only for illustrative purposes and does not constitute a structural limitation on the electronic device. In other embodiments of the present application, the electronic device may also adopt different interface connection methods or a combination of multiple interface connection methods in the above embodiments.

[0298] The external memory interface 320 may be used to connect to an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 310 through the external memory interface 320 to implement the data storage function. For example, files such as music and videos are saved in the external memory card.

[0299] The internal memory 321 can be used to store computer-executable program codes, and the executable program codes include instructions. The processor 310 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 321. The internal memory 321 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the electronic device (such as audio data, a phone book, etc.). In addition, the internal memory 321 can include a high-speed random access memory and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 310 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 321, and / or the instructions stored in the memory provided in the processor.

[0300] The wireless communication function of the electronic device can be implemented by antenna 1, antenna 2, the mobile communication module 350, the wireless communication module 360, the modulation and demodulation processor, and the baseband processor, etc.

[0301] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0302] The mobile communication module 350 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc. applied to the electronic device. The mobile communication module 350 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 350 can receive electromagnetic waves through antenna 1, perform processing such as filtering and amplifying on the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 350 can also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through antenna 1 for radiation. In some embodiments, at least some functional modules of the mobile communication module 350 can be provided in the processor 310. In some embodiments, at least some functional modules of the mobile communication module 350 and at least some modules of the processor 310 can be provided in the same device.

[0303] In some embodiments, the electronic device initiates or receives a call request through the mobile communication module 350 and antenna 1.

[0304] In addition, an operating system runs on the above components. For example, iOS operating system, Android operating system, Windows operating system, etc. Application programs can be installed and run on the operating system. Those skilled in the art can clearly understand that for the convenience and brevity of description, the explanations and beneficial effects of the relevant content in any of the above-mentioned electronic devices can refer to the corresponding method embodiments provided above, and will not be elaborated here.

[0305] This application also provides a communication system, which may include Figure 12 the first device as shown (such as a network device like a base station) and Figure 13 the second device as shown (such as a terminal like a mobile phone).

[0306] In this application, a terminal or a network device may include a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. Among them, the hardware layer may include hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also known as the main memory). The operating system in the operating system layer may be any one or more computer operating systems that implement service processing through processes. For example, Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system, etc. The application layer may include applications such as a browser, an address book, a word processing software, and an instant messaging software.

[0307] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and modules described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated here.

[0308] In several embodiments provided by this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of devices or modules can be in electrical, mechanical, or other forms.

[0309] The module described as a separation component may or may not be physically separated. The component shown as a module may or may not be a physical module, that is, it may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0310] In addition, each functional module in various embodiments of the present application can be integrated into a processing module, or each module can exist physically alone, or two or more modules can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module.

[0311] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the part that essentially contributes to the technical solution of the present application, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the processes of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories, random access memories, magnetic disks, or optical discs that can store program codes.

[0312] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of various embodiments of the present application.

Claims

1. A data synchronization method, characterized in that, The method includes: Obtaining the latency of a first data stream relative to a second data stream in a multi-modal service; When the latency of the first data stream relative to the second data stream is greater than or equal to a threshold, sending first information to a network device, where the first information is used to indicate that the latency of the first data stream relative to the second data stream is greater than or equal to the threshold; Receiving second information sent by the network device, where the second information includes resource configuration information for the first data stream and / or the second data stream; Determining, according to the second information, resources for transmitting the first data stream and / or resources for the second data stream.

2. The method according to claim 1, wherein The first information includes: The identifier of the first data stream and the identifier of the second data stream; or, At least one of the latency of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the identifier of the first data stream, the identifier of the second data stream; or, The identifier of a first logical channel and the identifier of a second logical channel, where the first logical channel is used to transmit the first data stream and the second logical channel is used to transmit the second data stream; or, At least one of the latency of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the identifier of the first logical channel, the identifier of the second logical channel; or, The identifier of a first data radio bearer (DRB) and the identifier of a second DRB, where the first DRB is used to transmit the first data stream and the second DRB is used to transmit the second data stream; or, At least one of the latency of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the identifier of the first DRB, the identifier of the second DRB; or, The modality of the first data stream and the modality of the second data stream; or, At least one of the latency of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the modality of the first data stream, the modality of the second data stream; or, An enumerated value used to indicate that the latency of the first data stream relative to the second data stream is greater than or equal to the threshold; or, At least one of the latency of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and an enumerated value, where the enumerated value is used to indicate that the latency of the first data stream relative to the second data stream is greater than or equal to a threshold.

3. The method according to claim 1 or 2, characterized in that, Sending the first information to the network device includes: Sending user equipment assistance information (UAI) to the network device, where the UAI includes the first information; or, Sending a media access control layer control element (MAC CE) to the network device, where the MAC CE includes the first information; or, Sending a packet data convergence protocol (PDCP) layer control message to the network device, where the PDCP layer control message includes the first information.

4. The method according to claim 3, wherein The UAI includes a stream information structure, and the stream information structure includes the synchronization information of the multimodal service, and the synchronization information is an array including the identifier of the first data stream and the identifier of the second data stream.

5. The method according to claim 4, characterized in that, The array further includes at least one of the latency of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, and the rate difference between the first data stream and the second data stream.

6. The method according to claim 3, wherein The MAC CE includes the identifier of the first logical channel and the identifier of the second logical channel.

7. The method according to claim 3, wherein The PDCP layer control message includes a type field and an identifier field, where the type field is used to indicate the enumerated value of the synchronization information, and the identifier field is used to indicate the identifier of the first data radio bearer (DRB) and the identifier of the second DRB.

8. The method according to any one of claims 1 to 7, characterized in that The method further includes: Sending the capability information of the terminal to the network device, where the capability information is used to indicate the support capability of the terminal for the synchronization of data streams in the multimodal service.

9. The method according to any one of claims 1 to 8, characterized in that, The threshold is configured by the network device, or the threshold is a predefined value.

10. The method according to any one of claims 1 to 9, characterized in that, The method further includes: Receiving the configuration information sent by the network device, where the configuration information is used to indicate the data streams to be synchronized, and the data streams to be synchronized include the first data stream and the second data stream.

11. The method according to any one of claims 1 to 10, characterized in that, The method is executed by a terminal, and the terminal is connected to multiple sensing devices, and each sensing device in the multiple sensing devices is used to collect at least one data stream of the multimodal service.

12. A data synchronization method, characterized in that, The method includes: Receiving first information from the terminal, where the first information is used to indicate that the latency of the first data stream relative to the second data stream in the multimodal service is greater than or equal to a threshold; Sending second information to the terminal, where the second information includes resource configuration information for the first data stream and / or the second data stream.

13. The method according to claim 12, characterized in that, The method further includes: Configuring resources for transmitting the first data stream and / or the second data stream, and obtaining the resource configuration information for the first data stream and / or the second data stream.

14. The method according to claim 12 or 13, characterized in that The first information includes: The identifier of the first data stream and the identifier of the second data stream; or, At least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the identifier of the first data stream, the identifier of the second data stream; or, The identifier of the first logical channel and the identifier of the second logical channel, where the first logical channel is used to transmit the first data stream and the second logical channel is used to transmit the second data stream; or, At least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the identifier of the first logical channel, the identifier of the second logical channel; or, The identifier of the first data radio bearer (DRB) and the identifier of the second DRB, where the first DRB is used to transmit the first data stream and the second DRB is used to transmit the second data stream; or, At least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the identifier of the first DRB, the identifier of the second DRB; or, The mode of the first data stream and the mode of the second data stream; or, At least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the mode of the first data stream, the mode of the second data stream; or, An enumerated value used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold; or, At least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the enumerated value used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold.

15. The method according to any one of claims 12 to 14, characterized in that Receiving the first information from the terminal includes: Receiving user equipment assistance information (UAI) from the terminal, where the UAI includes the first information; or, Receive a Medium Access Control layer control element MAC CE from a terminal, where the MAC CE includes the first information; or, Receive a Packet Data Convergence Protocol PDCP layer control message from a terminal, where the PDCP layer control message includes the first information.

16. An electronic device, characterized in that, The electronic device includes: A memory for storing a computer program or computer instructions; A processor for executing the computer program or computer instructions stored in the memory, so that the electronic device executes the method according to any one of claims 1 to 11.

17. An electronic device, characterized in that, The electronic device includes: A memory for storing a computer program or computer instructions; A processor for executing the computer program or computer instructions stored in the memory, so that the electronic device executes the method according to any one of claims 12 to 15.

18. A communication system, characterized in that, The system includes a first device and the second device, the first device is configured to execute the method according to any one of claims 1 to 11, and the second device is configured to execute the method according to any one of claims 12 to 15.

19. A computer storage medium for storing a computer program, which when executed is used to implement the method according to any one of claims 1 to 15.

Images