Data synchronization method and device

Abstract

The present application provides a data synchronization method and a related device. The method comprises: acquiring a delay of a first data stream with respect to a second data stream in a multi-modal service; if the delay of the first data stream with respect to the second data stream is greater than or equal to a threshold, sending first information to a network device, wherein the first information is used for indicating that the delay of the first data stream with respect to the second data stream is greater than or equal to the threshold; then receiving second information sent by the network device, the second information comprising resource configuration information for the first data stream and / or the second data stream; and then, on the basis of the second information, determining a resource for transmitting the first data stream and / or a resource for transmitting the second data stream. In the method, a terminal can monitor delays of different data streams in a multi-modal service, and when the delays are greater than or equal to a threshold, instructs a network device to configure resources of the data streams, thereby ensuring the synchronization of the different data streams, meeting service requirements, and enhancing user experience.

Description

Data synchronization method and device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on December 29, 2023, with application number 202311865232.9 and invention name “Data Synchronization Method and Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of communication technology, and in particular to a data synchronization method, electronic equipment, communication system, and computer-readable storage medium. Background Art

[0003] With the development of mobile communication technologies, especially the fifth-generation mobile networks (5G), the capabilities of communication systems are continuously enhanced. Specifically, 5G communication systems can provide enhanced mobile broadband (eMBB), with faster connections, higher throughput, and greater capacity, as well as ultra-reliable low-latency communications (uRLLC). This enables network applications in mission-critical scenarios that require uninterrupted and stable data links, such as extended reality (XR) and cloud gaming, meeting the ultra-high reliability and low latency requirements of wireless communication networks.

[0004] The services in the above scenarios may include multi-modal services. Multi-modal services refer to services that transmit data streams in multiple modes, which can be any combination of audio, video, tactile, and other modes. Taking the XR scenario as an example, XR services usually need to transmit data streams including audio, video, and tactile.

[0005] Currently, the network side usually adopts independent scheduling schemes for different data streams, which may cause a large delay for one data stream relative to another data stream in multimodal business scenarios. For example, the video data stream arrives a long time after the audio data stream, making it difficult to synchronize different data streams.

[0006] Summary of the Invention

[0007] The present application provides a data synchronization method and related equipment, the purpose of which is to solve the problem that in a multimodal business scenario, the network side adopts independent scheduling schemes for different data streams, which may cause a large delay of one data stream relative to another data stream, making it difficult to synchronize different data streams.

[0008] In order to achieve the above objectives, this application provides the following technical solutions:

[0009] The first aspect of the present application provides a data synchronization method. The method can be performed by a terminal. A terminal is also called a terminal device, a user unit, a user equipment (UE), a user station, a mobile station, a mobile station (MS), a mobile device, a user terminal, a wireless communication device, a user agent, or a user device. The terminal may include but is not limited to a mobile phone, a tablet computer (pad), a computer with wireless transceiver function, a virtual reality (VR) terminal, and an augmented reality (AR) terminal.

[0010] Specifically, the terminal can obtain the delay of the first data stream relative to the second data stream in the multimodal service. When the delay of the first data stream relative to the second data stream is greater than or equal to a threshold, first information is sent to the network device. The first information is used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to the threshold. The terminal then receives second information sent by the network device, and the second information includes resource configuration information for the first data stream and / or the second data stream. The terminal can determine the resources used to transmit the first data stream and / or the resources of the second data stream based on the second information.

[0011] In this method, the terminal can monitor the delay of different data streams in multimodal services. When the delay is greater than or equal to the threshold, the network device can be notified to configure the resources of the data stream to ensure the synchronization of different data streams, meet business needs, and improve user experience.

[0012] In some possible implementations, the first information includes:

[0013] an identifier of the first data stream and an identifier of the second data stream; or,

[0014] at least one of the following: a delay of the first data stream relative to the second data stream, a buffer size of the first data stream, a buffer size of the second data stream, a buffer difference between the first data stream and the second data stream, a rate of the first data stream, a rate of the second data stream, a rate difference between the first data stream and the second data stream, an identifier of the first data stream, and an identifier of the second data stream; or

[0015] an identifier of a first logical channel and an identifier of a second logical channel, the first logical channel being used to transmit a first data stream, and the second logical channel being used to transmit a second data stream; or

[0016] and at least one of the following: a delay of the first data stream relative to the second data stream, a buffer size of the first data stream, a buffer size of the second data stream, a buffer difference between the first data stream and the second data stream, a rate of the first data stream, a rate of the second data stream, a rate difference between the first data stream and the second data stream, an identifier of the first logical channel, and an identifier of the second logical channel; or,

[0017] An identifier of a first data radio bearer (DRB) and an identifier of a second DRB, the first DRB being used to transmit a first data stream, and the second DRB being used to transmit a second data stream; or

[0018] At least one of the following: the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, the identifier of the first DRB, and the identifier of the second DRB; or

[0019] the modality of the first data stream and the modality of the second data stream; or,

[0020] at least one of a delay of the first data stream relative to the second data stream, a buffer size of the first data stream, a buffer size of the second data stream, a buffer difference between the first data stream and the second data stream, a rate of the first data stream, a rate of the second data stream, a rate difference between the first data stream and the second data stream, a mode of the first data stream, and a mode of the second data stream; or

[0021] An enumeration value, where the enumeration value indicates that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold; or

[0022] at least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and an enumeration value, where the enumeration value is used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold.

[0023] In this method, the terminal can indicate that the delay is greater than or equal to the threshold through a variety of methods, such as the data stream identifier, logical channel identifier, DRB identifier, data stream mode, and enumeration value, which provides good flexibility and improves usability. Furthermore, when indicating that the delay is greater than or equal to the threshold, at least one of the specific delay, the buffer size of the compared data streams (e.g., the first data stream and the second data stream), the rate, the buffer difference, or the rate difference can also be indicated. This provides richer information for resource configuration on the network side.

[0024] In some possible implementations, the terminal may send User Equipment Assistance Information (UAI) to the network device, where the UAI includes the first information. Alternatively, the terminal may send a Medium Access Control Layer Control Element (MAC CE) to the network device, where the MAC CE includes the first information. Alternatively, the terminal may send a Packet Data Convergence Protocol (PDCP) layer control message to the network device, where the PDCP layer control message includes the first information.

[0025] In this method, the terminal can send the first information to the network side in different forms such as user equipment auxiliary information, MAC CE or PDCP layer control message, which has high flexibility and improves overall availability.

[0026] In some possible implementations, the UAI includes a stream information structure, which includes synchronization information for a multimodal service. The synchronization information is an array including an identifier of a first data stream and an identifier of a second data stream. This method reduces the impact on other services and the transmission overhead of sending the first information by reusing the stream information structure, such as the synchronization information within the stream information structure, to carry the first information.

[0027] In some possible implementations, the array further includes at least one of the following: a delay of the first data stream relative to the second data stream, a buffer size of the first data stream, a buffer size of the second data stream, a buffer difference between the first and second data streams, a rate of the first data stream, a rate of the second data stream, and a rate difference between the first and second data streams. By adding the delay of the first data stream relative to the second data stream to the array of synchronization information, this method can provide richer information for network-side resource configuration, enabling the network to better configure resources.

[0028] In some possible implementations, the MAC CE includes an identifier of the first logical channel and an identifier of the second logical channel. This method uses the MAC CE to send the identifier of the first logical channel and the identifier of the second logical channel, allowing the network side to perceive that the delay of the first data stream relative to the second data stream is greater than a threshold, and the network side can configure resources to ensure the synchronization of the data streams.

[0029] In some possible implementations, the PDCP layer control message includes a type field and an identification field. The type field is used to indicate an enumeration value of synchronization information, and the identification field is used to indicate the identification of the first DRB and the identification of the second DRB. This allows the network to detect that the delay of the first data stream relative to the second data stream is greater than a threshold, and the network can configure resources to ensure the synchronization of the data streams.

[0030] In some possible implementations, taking into account the capability differences between different terminals, the terminal may also send capability information of the terminal to the network device, and the capability information is used to indicate the terminal's ability to support data stream synchronization in multimodal services. The terminal's ability to support data stream synchronization in multimodal services may include supporting data stream synchronization in multimodal services, or not supporting data stream synchronization in multimodal services. By reporting capability information, the terminal can enable the network device to send resource configuration information to the terminal that supports data stream synchronization in multimodal services in a targeted manner. For terminals that do not support data stream synchronization in multimodal services, the network device can use the original method for resource scheduling and data transmission. This can improve overall compatibility.

[0031] In some possible implementations, the threshold value can be configured by the network device, or the threshold value can be a predefined value. The network device configuration of the threshold value can achieve real-time configuration and can adjust the threshold value according to business needs, thereby achieving better resource allocation results.

[0032] In some possible implementations, a terminal can receive configuration information sent by a network device. The configuration information indicates the data streams to be synchronized, including a first data stream and a second data stream. This allows monitoring of the latency of specific data streams configured on the network side to meet personalized service needs.

[0033] In some possible implementations, the method can be performed by a terminal. The terminal is connected to multiple sensor devices, such as XR devices such as XR glasses or controllers. Each of the multiple sensor devices is configured to collect at least one data stream of a multimodal service. This allows for resource allocation for multimodal services based on the same terminal, ensuring synchronization of data streams within the multimodal service.

[0034] A second aspect of the present application provides a data synchronization method, which can be executed by a network device. The network device can be an access network device, such as a base station, a wireless access point, a transmission and reception point, a wireless relay node, a wireless backhaul node, etc.

[0035] Specifically, the network device receives first information from the terminal, where the first information is used to indicate that the delay of the first data stream relative to the second data stream in the service is greater than or equal to a threshold, and then sends second information to the terminal, where the second information includes resource configuration information for the first data stream and / or the second data stream.

[0036] In this method, the network device can configure the resources used to transmit the first data stream and / or the second data stream based on the first information sent by the terminal to indicate the delay timeout of the first data stream relative to the second data stream in the multimodal service, so as to ensure the synchronization of data streams of different modes, meet business needs, and improve user experience.

[0037] In some possible implementations, the network device may further configure resources for transmitting the first data stream and / or the second data stream, and obtain the resource configuration information for the first data stream and / or the second data stream. For example, the network device may configure resources for transmitting the first data stream and / or the second data stream at the data stream granularity or at the multimodal service granularity to shorten the latency of the first data stream relative to the second data stream and ensure synchronization between the first data stream and the second data stream.

[0038] In some possible implementations, the first information includes:

[0039] an identifier of the first data stream and an identifier of the second data stream; or,

[0040] at least one of the following: a delay of the first data stream relative to the second data stream, a buffer size of the first data stream, a buffer size of the second data stream, a buffer difference between the first data stream and the second data stream, a rate of the first data stream, a rate of the second data stream, a rate difference between the first data stream and the second data stream, an identifier of the first data stream, and an identifier of the second data stream; or

[0041] an identifier of a first logical channel and an identifier of a second logical channel, the first logical channel being used to transmit a first data stream, and the second logical channel being used to transmit a second data stream; or

[0042] at least one of the following: a delay of the first data stream relative to the second data stream, a buffer size of the first data stream, a buffer size of the second data stream, a buffer difference between the first data stream and the second data stream, a rate of the first data stream, a rate of the second data stream, a rate difference between the first data stream and the second data stream, an identifier of the first logical channel, and an identifier of the second logical channel; or

[0043] An identifier of a first data radio bearer (DRB) and an identifier of a second DRB, the first DRB being used to transmit a first data stream, and the second DRB being used to transmit a second data stream; or

[0044] At least one of the following: the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, the identifier of the first DRB, and the identifier of the second DRB; or

[0045] the modality of the first data stream and the modality of the second data stream; or,

[0046] at least one of a delay of the first data stream relative to the second data stream, a buffer size of the first data stream, a buffer size of the second data stream, a buffer difference between the first data stream and the second data stream, a rate of the first data stream, a rate of the second data stream, a rate difference between the first data stream and the second data stream, a mode of the first data stream, and a mode of the second data stream; or

[0047] An enumeration value, where the enumeration value indicates that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold; or

[0048] at least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and an enumeration value, where the enumeration value is used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold.

[0049] In some possible implementations, receiving first information from a terminal includes:

[0050] receiving user equipment assistance information UAI from a terminal, where the UAI includes the first information; or

[0051] receiving a medium access control layer control element MAC CE from a terminal, where the MAC CE includes the first information; or,

[0052] A Packet Data Convergence Protocol (PDCP) layer control message is received from a terminal, where the PDCP layer control message includes the first information.

[0053] In this method, the network device supports obtaining the first information in different forms such as user equipment auxiliary information, MAC CE or PDCP layer control message, which has high flexibility and improves overall availability.

[0054] The third aspect of the present application provides an electronic device, comprising: a memory and at least one processor. The memory is used to store programs, and the at least one processor is used to run the programs, so that the electronic device implements the data synchronization method provided in the first aspect of the present application.

[0055] A fourth aspect of the present application provides an electronic device comprising: a memory and at least one processor. The memory is used to store programs, and the at least one processor is used to run the programs, so that the electronic device implements the data synchronization method provided in the second aspect of the present application.

[0056] A fifth aspect of the present application provides a communication system, including a first device and a second device. The first device and the second device are configured to execute the data synchronization method provided in the third aspect or the fourth aspect of the present application.

[0057] The sixth aspect of the present application is a computer storage medium for storing a computer program. When the computer program is executed, it is used to implement the data synchronization method provided by the first aspect or the second aspect of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] FIG1 is an exemplary diagram of a communication system architecture disclosed in an embodiment of the present application;

[0059] FIG2 is a diagram of an application scenario disclosed in an embodiment of the present application;

[0060] FIG3 is a flow chart of a data synchronization method disclosed in an embodiment of the present application;

[0061] FIG4 is a schematic diagram of determining the delay of different data streams disclosed in an embodiment of the present application;

[0062] FIG5 is a schematic diagram of the format of a media access control layer control element disclosed in an embodiment of the present application;

[0063] FIG6 is a schematic diagram of the format of another medium access control layer control element disclosed in an embodiment of the present application;

[0064] FIG7 is a schematic diagram of the format of another medium access control layer control element disclosed in an embodiment of the present application;

[0065] FIG8 is a schematic diagram of the format of a packet data convergence protocol layer control message disclosed in an embodiment of the present application;

[0066] FIG9 is a flow chart of a data synchronization method disclosed in an embodiment of the present application;

[0067] FIG10 is a schematic diagram of an application scenario in which data synchronization is achieved by sending first information through a radio resource control message disclosed in an embodiment of the present application;

[0068] FIG11 is a schematic diagram of an application scenario in which data synchronization is achieved by sending first information through a media access control layer control element, as disclosed in an embodiment of the present application;

[0069] FIG12 is a structural diagram of an electronic device disclosed in an embodiment of the present application;

[0070] FIG13 is a structural diagram illustrating another electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to be limiting of the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the embodiments of the present application, "one or more" refers to one, two or more; "and / or" describes the association relationship of associated objects, indicating that three relationships may exist; for example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship.

[0072] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0073] The "multiple" involved in the embodiments of the present application means greater than or equal to two. It should be noted that in the description of the embodiments of the present application, the words "first" and "second" are only used for the purpose of distinguishing the description and cannot be understood as indicating or implying relative importance or order.

[0074] The technical solutions of the embodiments of the present application can be applied to various communication systems, such as: global system for mobile communications (GSM) system, code division multiple access (CDMA) system, wideband code division multiple access (WCDMA) system, general packet radio service (GPRS), long term evolution (LTE) system, LTE frequency division duplex (FDD) system, LTE time division duplex (TDD), universal mobile telecommunication system (UMTS), worldwide interoperability for microwave access (WiMAX) communication system, fifth generation (5G) system or new radio (NR), and future communication systems.

[0075] To meet the challenges of wireless broadband technology and maintain the leading edge of 3GPP networks, the 3GPP standards group has developed a next-generation mobile communications network architecture, known as the 5G network architecture. This architecture not only supports 3GPP-defined wireless technologies (such as LTE) accessing the 5G core network (5GC), but also supports non-3GPP access technologies accessing the 5GC through the non-3GPP interworking function (N3IWF), the trusted non-3GPP gateway function (TNGF), the trusted WLAN interworking function (TWIF), or the next-generation packet data gateway (NG-PDG). The core network functions are divided into user plane function (UPF) and control plane function (CPF). The UPF is primarily responsible for packet forwarding, quality of service (QoS) control, and billing information statistics. CPF is mainly responsible for user registration and authentication, mobility management, and sending data packet forwarding policies and QoS control policies to UPF. It can be further divided into access and mobility management function (AMF) and session management function (SMF).

[0076] Core network equipment includes, for example, a mobility management entity (MME), a broadcast multicast service center (BMSC), etc., or may also include corresponding functional entities in the 5G system, such as core network control plane (CP) or user plane (UP) network functions, such as SMF, AMF, etc. The core network control plane can also be understood as a core network control plane function (CPF) entity.

[0077] FIG1 is an example of a communication system architecture applicable to an embodiment of the present application, wherein the functions of the user equipment and each network entity are as described below.

[0078] Terminal: can be called terminal equipment, terminal equipment unit (subscriber unit), terminal equipment station, terminal equipment agent, terminal equipment device, access terminal, terminal in V2X communication, subscriber unit, user equipment (UE), subscriber station, mobile station, mobile station (MS), remote station, remote terminal, mobile device, user terminal, wireless communication device, user agent or user device.

[0079] The user equipment in the embodiments of the present application may also be a mobile phone, a tablet computer, a computer with wireless transceiver function, a holographic projector, a video player, a virtual reality (VR) terminal, an augmented reality (AR) terminal, a wireless terminal in industrial control, a tactile terminal device, a vehicle-mounted terminal device, a wireless terminal in self-driving, a wireless terminal in remote medical, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a wearable device, a terminal in a 5G network or a terminal in a future evolution network, etc.

[0080] Wearable devices, also known as wearable smart devices, are a general term for wearable devices developed by applying wearable technology to intelligently design everyday wearables. These include, but are not limited to, XR glasses (such as AR glasses or VR glasses), gloves, watches, clothing, and shoes. A wearable device is a portable device worn directly on the body or integrated into a user's clothing or accessories. Wearable devices are not merely hardware devices; they achieve powerful functionality through software support, data interaction, and cloud-based interaction. Broadly speaking, wearable smart devices include those that are full-featured, large in size, and can function completely or partially independently of a smartphone, such as smartwatches or smart glasses, as well as those that focus on a specific application and require integration with other devices, such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring. It should be noted that wearable devices (such as XR glasses and other XR devices) can also integrate smartphone functionality. For example, XR devices can also integrate a Subscriber Identity Module (SIM) or an embedded SIM (eSIM) for cellular communication.

[0081] Radio access network (RAN): A network composed of multiple 5G-RAN nodes that implements wireless physical layer functions, resource scheduling and wireless resource management, wireless access control, and mobility management. 5G-RAN is connected to the UPF via the user plane interface N3 to transmit data from terminal devices. 5G-RAN establishes a control plane signaling connection with the AMF via the control plane interface N2 to implement functions such as radio access bearer control. RAN can be any device with wireless transceiver capabilities, including but not limited to 5G base stations (5G node base, gNB), evolutionary node base (eNB), wireless access points (WiFi AP), world interoperability for microwave access base stations (WiMAX BS), transmission receiving points (TRP), wireless relay nodes, wireless backhaul nodes, etc.

[0082] The access network device (i.e., the network device of the access network) in the embodiment of the present application can also be a device for communicating with a terminal device. The access network device can be a base station (base transceiver station, BTS) in a global system of mobile communication (GSM) system or code division multiple access (CDMA), or a base station (nodeB, NB) in a wideband code division multiple access (WCDMA) system, or an evolutionary node base (eNB) in an LTE system, or a wireless controller in a cloud radio access network (CRAN) scenario, or the access network device can be a relay station, an access point, a vehicle-mounted device, a wearable device, and an access network device in a future 5G network or an access network device in a future evolved PLMN network, etc., and the embodiment of the present application is not limited.

[0083] In NR, the functions of the base station are divided into two parts, called centralized unit (CU)-distributed unit (DU) separation. From the perspective of the protocol stack, the CU includes the RRC layer and PDCP layer of the LTE base station, and the DU includes the radio link control (RLC) layer, media access control (MAC) layer and physical (PHY) layer of the LTE base station. In ordinary 5G base station deployments, the CU and DU can be physically connected through optical fiber, and logically there is a specially defined F1 interface for communication between the CU and the DU. From a functional perspective, the CU is mainly responsible for wireless resource control and configuration, cross-cell mobility management, bearer management, etc. The DU is mainly responsible for scheduling, physical signal generation and transmission.

[0084] Among them, the above-mentioned base stations can be macro base stations, micro base stations, pico base stations, small stations, relay stations, balloon stations, etc.

[0085] SMF: Mainly responsible for the control plane functions of terminal device session management, including selection and control of user plane functions (UPF), Internet protocol (IP) address allocation, session QoS management, and obtaining policy and charging control (PCC) policies (from PCF).

[0086] UPF: As the anchor point for protocol data unit (PDU) session connections, it is responsible for filtering data packets for terminal devices, data transmission / forwarding, rate control, generating billing information, etc., and providing connections to the data network (DN).

[0087] PCF: Provides configuration policy information for terminal devices and policy information for controlling terminal devices to network control plane elements (such as SMF); generates terminal device access policies and QoS flow control policies.

[0088] AF: interacts with network elements in the core network to provide some services. For example, it interacts with PCF to perform service policy control, interacts with NEF to obtain some network capability information or provide some application information to the network, and provides some data network access point information to PCF to generate routing information for corresponding data services.

[0089] In the embodiments of the present application, the terminal device is wirelessly connected to the RAN device, and the RAN network element is wirelessly or wiredly connected to the 5GC device. The 5GC device and the RAN network element can be independent and distinct physical devices, or the functions of the 5GC device and the logical functions of the RAN network element can be integrated into the same physical device, or a single physical device can integrate some of the functions of the 5GC device and some of the functions of the RAN network element. The terminal device can be fixed or mobile.

[0090] 5GC equipment mainly includes the above-mentioned PCF network elements, SMF network elements and UPF network elements.

[0091] It should be noted that the aforementioned "network element" may also be referred to as an entity, device, apparatus, or module, and this application does not specifically limit this. Furthermore, in this application, for ease of understanding and explanation, the term "network element" is omitted in some descriptions. For example, an NEF network element is referred to as NEF. In this case, the "NEF" should be understood as either an NEF network element or an NEF entity. The following descriptions of identical or similar situations are omitted.

[0092] It should be noted that the naming of each network element included in Figure 1 is only a name, and the name does not limit the function of the network element itself. In 5G networks and other future networks, the above-mentioned network elements may also have other names, and the embodiments of this application do not specifically limit this. For example, in a 6G network, some or all of the above-mentioned network elements may continue to use the terminology used in 5G, or may have other names, etc., which are uniformly explained here and will not be repeated below.

[0093] It should be noted that the network elements in Figure 1 do not have to exist simultaneously, and the network elements required can be determined according to needs. The connection relationship between the network elements in Figure 1 is not unique and can be adjusted according to needs.

[0094] It is understandable that the above-mentioned network elements or functions can be network elements in hardware devices, software functions running on dedicated hardware, or virtualized functions instantiated on a platform (for example, a cloud platform).

[0095] Figure 2 is a schematic diagram of an application scenario applicable to the present application provided by the present application. As shown in Figure 2, an embodiment of the present application can be applied to a multimodal business scenario. The multimodal business scenario may include an AF network element and an XR device. Among them, the AF network element may be an XR server, and the XR device may be an XR head-mounted display (HMD) or XR glasses, wherein the XR device may also be configured with a controller (such as a handle). The XR server may receive video frames (video frame), audio data (audio data), haptic data (haptic data) or posture and control (pose & control) data on the XR device side, and return processed video frames, audio data, haptic data or posture and control data to the XR device. Among them, the XR server and the XR device exchange various commands and feedback signals through the communication link in the network architecture to form a global control loop.

[0096] In multimodal service applications, multiple data flows (such as QoS flows) are required to transmit different data types, such as video, audio, haptics, or gesture and control. Each QoS flow can be used to transmit each data type. In this example, the multimodal service includes four service types, so four QoS flows can be used for data transmission.

[0097] The various data streams of a multimodal service are correlated in space and time. Spatial correlation refers to the correlation between different data streams at the same position in space, and temporal correlation refers to the correlation between different data streams over a period of time. The following examples illustrate spatial correlation and temporal correlation respectively. For example, when a human body touches an object with different surface textures and different materials, the tactile sensation is also different. There is a certain correlation between tactile data and the surface image of the object in the video data, and this correlation is also a specific form of spatial correlation. The spatial correlation between different data streams can be used to assist in data reconstruction. For example, when tactile data is transmitted over the network, some data packets are lost due to channel fluctuations or network congestion. At this time, the spatial correlation of different data streams in the multimodal service can be used to restore and reconstruct the damaged data. In this example, video data can be used to restore tactile data to ensure the user experience of the multimodal service.

[0098] In other examples, when a person is speaking, the lip shape of the person in the video data may change. The audio data of a time period has a certain correlation with the lip shape image in the video data before and after the time period. This correlation is also a specific form of time correlation. The time correlation between different data streams can be used to assist in data reconstruction. For example, when audio data is transmitted over the network and some data packets are lost due to channel fluctuations or network congestion, the time correlation of different data streams in the multimodal service can be used to restore and reconstruct the damaged data. In this example, the video data can be used to restore the audio data to ensure the user experience of the multimodal service.

[0099] In the example of Figure 2, the data collected by the XR device, such as video frames, audio data, tactile data, posture and control data, can be transmitted to the access network through the terminal (such as UE), for example, to the access network device (such as gNB), wherein the terminal can map different types of data to corresponding bearers respectively, and the data transmitted on each bearer can form a QoS flow (QoS flow). The access network device can transmit the above data to 5GC in the form of QoS flow, and then 5GC transmits the above data to the XR server. It should be noted that the XR device can also communicate directly with the access network device. For example, when the XR device integrates eSIM, or the functions of the UE are integrated into the XR device, the XR device can be directly connected to the access network device. Accordingly, the XR device can transmit the collected data directly to the access network device, for example, directly to the access network device in the form of QoS flow without passing through the UE.

[0100] Correspondingly, the XR server can return a multimodal data stream to the XR device. Specifically, the XR server can first return the multimodal data stream to the 5GC, which then transmits the multimodal data stream to the access network device in the form of a QoS stream. The access network device then transmits the multimodal data stream to the corresponding UE, and the UE then transmits the received data stream to the XR device.

[0101] It should be noted that Figure 2 illustrates a multimodal scenario where an XR device is connected to the network through a single terminal, such as a multimodal scenario based on the same UE or intra-UE. Multiple XR devices can connect to the network through the same UE, which can serve as a unified entry or exit for multiple XR devices to interact with the XR server.

[0102] In addition to the application scenarios constructed between the application server and the terminal device described above, the application scenarios of this application can also be constructed by multiple terminal devices. In the scenario constructed by multiple terminal devices, one terminal device can be an XR device and another terminal device can be a remote robot, and the XR device can control the operation of the remote robot. The construction of the application scenarios of this application is not limited to the above scenarios. All application scenarios that can be applied to the technical solution of this application are within the scope of protection of this application.

[0103] It should be noted that when a communication system requires multimodal services, the SMF establishes corresponding PDU sessions. Generally, one PDU session is established for each multimodal service. A PDU session can correspond to multiple radio bearers (RBs), and a PDU session can support multiple data flows with different QoS requirements.

[0104] The synchronization between data streams significantly impacts the user experience. For example, if the audio and video data streams are out of sync, the lip movements in the video may not match the audio. Currently, the network typically uses independent scheduling for different data streams. This results in significant latency between one data stream and another in multimodal service scenarios. For example, the video data stream may arrive long after the audio data stream, making synchronization difficult.

[0105] In view of this, the present application provides a data synchronization method. Specifically, the terminal can obtain the delay of the first data stream (such as QoS flow1 for transmitting video data) relative to the second data stream (such as QoS flow2 for transmitting tactile data) in a multimodal service. When the delay of the first data stream relative to the second data stream is greater than or equal to a threshold, the terminal can send a first information to a network device (such as an access network device, including but not limited to a base station), and the first information can explicitly or implicitly indicate that the delay of the first data stream relative to the second data stream is greater than or equal to the threshold. Then, the terminal receives the second information sent by the network device, and the second information includes resource configuration information of the first data stream and / or the second data stream. The terminal can determine the resources for transmitting the first data stream and / or the resources for transmitting the second data stream based on the resource configuration information.

[0106] In this method, the terminal can monitor the delay of different data streams in multimodal services. When the delay is greater than or equal to the threshold, the network device can be notified to configure the resources of the data stream to ensure the synchronization of different data streams, meet business needs, and improve user experience.

[0107] In order to make the technical solution of the present application clearer and easier to understand, the data synchronization method of the embodiment of the present application is introduced below with reference to the accompanying drawings.

[0108] Referring to the flowchart of a data synchronization method shown in FIG3 , the method includes:

[0109] S302: The terminal obtains the delay of the first data stream relative to the second data stream in the multimodal service. If the delay of the first data stream relative to the second data stream is greater than or equal to a threshold, S304 is executed. The threshold corresponds to the service type of the first data stream and the service type of the second data stream.

[0110] Multimodal services may include multiple interrelated data streams, and multiple data streams may come from different data sources. Each data stream can be regarded as a type (modality) of data stream associated with the same service, such as an audio data stream, a video data stream, a position data stream, or a tactile data stream. Each data stream in a multimodal service is also referred to as a multimodal service stream. In some examples, multimodal services may include but are not limited to XR services and cloud gaming services.

[0111] The first data stream and the second data stream can be different data streams in a multimodal service. Typically, the first data stream and the second data stream can be data streams of different modalities or different service types. For example, the first data stream can be a video data stream, and the second data stream can be a tactile data stream. Taking the example of a user playing a cloud game through an XR device, the first data stream can be a video data stream formed by video data collected by the XR glasses, and the second data stream can be a tactile data stream formed by tactile data collected by the controller.

[0112] The time delay of a first data stream relative to a second data stream can be the time difference between the first data stream and the second data stream. Data packets in a data stream typically include a timestamp. The timestamp reflects the sampling moment of the first byte of data in a Real-time Transport Protocol (RTP) packet. The initial timestamp value at the start of a session is also randomly selected. If two streams start sampling at the same time, the RTP timestamp can be used to synchronize the data streams. As shown in Figure 4, the starting time of the two data streams is the same. Based on the sampling period, the absolute time value of each frame can be determined, and the time difference (i.e., time delay) between the two data streams can be determined.

[0113] In a specific implementation, the terminal can obtain the first data packet in multiple buffers of the Packet Data Convergence Protocol (PDCP) layer. The buffer can be a queue, and the queue includes multiple elements, each of which can store a data packet. Generally, the data packets of a data stream can be stored in one buffer. The data packets of different data streams can be stored in different buffers. In some examples, multiple data streams can also be stored in one buffer. Different data streams in the same buffer can be identified by port numbers. For ease of description, the following takes the data packets of a data stream stored in one buffer as an example.

[0114] Among them, the PDCP layer belongs to the second layer of the radio interface protocol stack, which is used to process Radio Resource Control (RRC) messages on the control plane and Internet Protocol (IP) packets on the user plane. The data packets in the buffer of the PDCP layer can be service data units (SDUs). SDU, also known as service data unit, is a data set of user services of a specified layer. When it is transmitted to the receiver, the data of the same protocol layer (i.e., the service part) does not change. When the data reaches the lower layer, the lower layer encapsulates the data in a protocol data unit (PDU) for transmission. The terminal can obtain the RTP header of the SDU, extract the timestamp from the RTP header, and determine the delay of different data streams based on the timestamp extracted from the RTP header of the first SDU in different buffers. For example, the terminal can determine the delay of the first data stream relative to the second data stream in the above manner. The delay can be the difference determined based on the timestamp extracted from the RTP header of the first SDU in different buffers, also known as the synchronization time difference.

[0115] The terminal can compare the delay of the first data stream relative to the second data stream and the threshold value corresponding to the service type of the first data stream and the service type of the second data stream (such as the threshold value of the first service type relative to the second service type). The synchronization requirements of data streams of different service types may be different, and accordingly, the threshold values ​​corresponding to data streams of different service types (also referred to as synchronization delay thresholds, synchronization thresholds) may be different. The threshold value may be configured by a network device or predefined by a protocol, and accordingly, the threshold value may be a value predefined in the protocol. In some possible implementations, the mapping relationship between the service type and the threshold value may be predefined, and the terminal may store the mapping relationship between the service type and the synchronization delay threshold value to facilitate subsequent comparison of the delays and threshold values ​​of different data streams.

[0116] The terminal can store the mapping relationship between service types and thresholds in a table, as shown below:

[0117] Table 1 Mapping relationship between service types and thresholds

[0118] In Table 1, "audio" represents audio, and "tactile" represents tactile. Using the example of audio and tactile, audio delay represents the maximum delay of audio relative to tactile. This means that after tactile arrives, audio should arrive no later than this maximum delay. If it arrives after this maximum delay, audio and tactile are out of sync. Tactile delay represents the maximum delay of tactile relative to audio. This means that after audio arrives, tactile should arrive no later than this maximum delay. If it arrives after this maximum delay, tactile and audio are out of sync.

[0119] When the time delay of the first data stream relative to the second data stream is greater than or equal to the threshold of the corresponding type, the operation of S304 can be triggered. The following uses audio and tactile examples to illustrate that when the terminal obtains the time delay of the audio type data stream relative to the tactile type data stream, the time delay can be compared with the threshold of audio relative to tactile (such as 50ms). If the time delay of the audio type data stream relative to the tactile type data stream is greater than or equal to the threshold of audio relative to tactile, the operation of S304 can be triggered.

[0120] It should be noted that the terminal can also receive configuration information sent by the network device. The configuration information can indicate information about the data stream to be synchronized. The data stream to be synchronized can include a first data stream and a second data stream. The configuration information can include at least one of the following: an identifier of the data stream to be synchronized, an identifier of the logical channel used by the data stream to be synchronized, a mode of the data stream to be synchronized, and an identifier of the data radio bearer of the data stream to be synchronized. Accordingly, the terminal can obtain the delay of the data stream indicated by the configuration information based on the above configuration information. For example, the configuration information can include an identifier of the first data stream and an identifier of the second data stream, and the terminal can obtain the delay of the first data stream relative to the second data stream based on the configuration information. Furthermore, the configuration information can also include a threshold corresponding to the service type of the data stream to be synchronized, or a threshold corresponding to the service type to be synchronized. In this way, the terminal can compare the delay of the data stream to be synchronized and the threshold corresponding to the data stream to be synchronized based on the configuration information.

[0121] It should also be noted that the multimodal service in this embodiment can be a single-terminal multimodal service. The terminal is connected to multiple sensor devices (such as XR devices or other sensor devices), and each of the multiple sensor devices is used to collect data streams of at least one modality. The terminal can obtain the delay of the first data stream relative to the second data stream in the above-mentioned multimodal service, and compare the delay and the threshold corresponding to the service type of the above-mentioned data stream (such as the threshold corresponding to the service type of the first data stream and the service type of the second data stream).

[0122] S304: The terminal sends first information to the network device.

[0123] The first information may explicitly or implicitly indicate that the time delay of the first data stream relative to the second data stream is greater than or equal to a threshold. The first data stream and the second data stream may be data streams of different modes in a multimodal service. In some cases, the first data stream and the second data stream may be video streams of different modes. For example, the first data stream may be a video stream, and the second data stream may be an audio stream. In other cases, the first data stream and the second data stream may be video streams of the same mode. For example, the first data stream and the second data stream may both be video streams. The first information may indicate that the time delay of the first data stream relative to the second data stream is greater than or equal to a threshold through different implementation methods. The different implementation methods are described in detail below.

[0124] In a first implementation, the first information may include an identifier of the first data flow and an identifier of the second data flow. The first data flow and the second data flow may be transmitted in the form of QoS flows, and accordingly, the identifier of the first data flow and the identifier of the second data flow may be QoS flow identifiers (QFIs), such as QFI 1 and QFI 2.

[0125] In this implementation, the first information may be sent to the network device in the form of an array or a sequence. For example, the first information may be in the following format:

[0126] First Information

[0127] Data flow sequence {QFI 1, QFI 2}

[0128] }

[0129] Among them, QFI 1 and QFI 2 are identifiers of data streams. It should be noted that, in this example, sequence{QFI 1, QFI 2} indicates that the delay of data stream 1 relative to data stream 2 is greater than or equal to a threshold. It should be noted that the terminal can first negotiate the format of the sequence with the network device, or agree on the format of the sequence through a protocol. For example, the format of the sequence can be such that the identifiers of the data streams that arrive later are arranged in front, and the identifiers of the data streams that arrive first are arranged in the back. In other possible implementation methods of the embodiments of the present application, sequence{QFI 1, QFI 2} can also indicate that the delay of data stream 2 relative to data stream 1 is greater than or equal to a threshold. In this case, if there is a delay in data stream 1 relative to data stream 2, it can be indicated by sequence{QFI 2, QFI 1}.

[0130] In a second implementation, the first information may include an identifier of the first data stream, an identifier of the second data stream, and a delay of the first data stream relative to the second data stream. The delay may be determined by a system frame number (SFN), a subframe, a time slot, or absolute time, and the delay unit may be a symbol, a time slot, a subframe, a frame, or a millisecond. For ease of description, the following example uses an example where the delay is determined by absolute time and the delay unit is milliseconds.

[0131] In this implementation, the first information may be sent to the network device in the form of an array or a sequence. For example, the first information may be in the following format:

[0132] First Information

[0133] Data flow sequence {QFI 1, QFI 2}

[0134] Delay sequence {t delay}

[0135] }

[0136] Among them, QFI 1 and QFI 2 are the identifiers of the data flow, t delay is the delay in milliseconds.

[0137] This example uses two arrays to provide the data stream identifier and delay respectively. In actual applications, you can also use a single array to provide the data stream identifier and delay, as shown below:

[0138] First Information

[0139] Data flow and delay sequence {QFI 1, QFI 2, t delay}

[0140] }

[0141] In a third implementation, the first information may include a modality of the first data stream and a modality of the second data stream. The modality may be identified by a modality name or a modality type. For example, the modality of the first data stream may be video (or visual), and the modality of the second data stream may be tactile.

[0142] In this implementation, the first information may be sent to the network device in the form of an array or a sequence. For example, the first information may be in the following format:

[0143] First Information

[0144] Modal sequence{modal type 1, modal type 2}

[0145] }

[0146] Among them, modal type 1 and modal type 2 are the modality of the first data stream and the modality of the second data stream (specifically the first modality and the second modality).

[0147] In a fourth implementation, the first information may include the mode of the first data stream, the mode of the second data stream, and the delay of the first data stream relative to the second data stream.

[0148] In this implementation, the first information may be sent to the network device in the form of an array or a sequence. For example, the first information may be in the following format:

[0149] First Information

[0150] Modal sequence{modal type 1, modal type 2}

[0151] Delay sequence {t delay}

[0152] }

[0153] Among them, modal type 1 and modal type 2 are modal, t delay The delay is, for example, the delay of the first data stream relative to the second data stream.

[0154] This example uses two arrays to provide the mode and delay respectively. In actual applications, you can also use one array to provide the mode and delay, as shown below:

[0155] First Information

[0156] Modal and delay sequence {modal type 1, modal type 2, t delay}

[0157] }

[0158] In a fifth implementation, the first information may include an identifier of a first logical channel and an identifier of a second logical channel, wherein the first logical channel is used to transmit the first data stream, and the second logical channel is used to transmit the second data stream.

[0159] In this implementation, the first information may be sent to the network device in the form of an array or a sequence. For example, the first information may be in the following format:

[0160] First Information

[0161] Logical channel sequence {LCID 1, LCID 2}

[0162] }

[0163] Among them, LCID 1 and LCID 2 are identifiers (IDs) of logical channels. It should be noted that, in this example, sequence{LCID 1, LCID 2} indicates that the delay of data stream 1 in logical channel 1 relative to data stream 2 in logical channel 2 is greater than or equal to a threshold. In other possible implementations of the embodiments of the present application, sequence{LCID 1, LCID 2} may also indicate that the delay of data stream 2 in logical channel 2 relative to data stream 1 in logical channel 1 is greater than or equal to a threshold. In this case, if there is a delay in data stream 1 relative to data stream 2, it can be indicated by sequence{LCID 2, LCID 1}.

[0164] In a sixth implementation manner, the first information may include an identifier of the first logical channel, an identifier of the second logical channel, and a delay of the first data stream relative to the second data stream.

[0165] In this implementation, the first information may be sent to the network device in an array or sequence. For example, the first information may be in the following format:

[0166] First Information

[0167] Logical channel sequence {LCID 1, LCID 2}

[0168] Delay sequence {t delay}

[0169] }

[0170] Among them, LCID 1 and LCID 2 are the identifiers of the logical channels, t delay is the delay in milliseconds.

[0171] This example uses two arrays to provide the logical channel identifier and delay respectively. In actual applications, you can also use a single array to provide the logical channel identifier and delay, as shown below:

[0172] First Information

[0173] Logical channel and delay sequence {LCID 1, LCID 2, t delay}

[0174] }

[0175] In a seventh implementation, the first information may include an identifier of a first data radio bearer (DRB) and an identifier of a second DRB, wherein the first DRB is used to transmit the first data stream and the second DRB is used to transmit the second data stream.

[0176] In this implementation, the first information may be sent to the network device in the form of an array or a sequence. For example, the first information may be in the following format:

[0177] First Information

[0178] Data radio bearer sequence {DRB_ID 1, DRB_ID 2}

[0179] }

[0180] Among them, DRB_ID 1 and DRB_ID 2 are the identifiers of DRBs. It should be noted that, in this example, sequence{DRB_ID 1, DRB_ID 2} indicates that the delay of data stream 1 carried by DRB_ID 1 relative to data stream 2 carried by DRB_ID 2 is greater than or equal to a threshold. In other possible implementations of the embodiments of the present application, sequence{DRB_ID 1, DRB_ID 2} may also indicate that the delay of data stream 2 carried by DRB_ID 2 relative to data stream 1 carried by DRB_ID 1 is greater than or equal to a threshold. In this case, if there is a delay in data stream 1 relative to data stream 2, it can be indicated by sequence{DRB_ID 2, DRB_ID 1}.

[0181] In an eighth implementation, the first information may include an identifier of the first DRB, an identifier of the second DRB, and a delay of the first data stream relative to the second data stream.

[0182] In this implementation, the first information may be sent to the network device in the form of an array or a sequence. For example, the first information may be in the following format:

[0183] First Information

[0184] Data radio bearer sequence {DRB_ID 1, DRB_ID 2}

[0185] Delay sequence {t delay}

[0186] }

[0187] This example uses two arrays to provide the DRB identifier and delay respectively. In actual applications, you can also use a single array to provide the DRB identifier and delay, as shown below:

[0188] First Information

[0189] Data radio bearer and delay sequence {DRB_ID 1, DRB_ID 2, t delay}

[0190] }

[0191] In a ninth implementation, the first information includes an enumeration value that indicates that the latency of the first data stream relative to the second data stream is greater than or equal to a threshold. Specifically, an enumeration is a user-defined type consisting of multiple comma-delimited string constants. These constants are referred to as enumeration values.

[0192] In this application, users can define string constants 1, 2, ..., n, which respectively indicate that the delay of a data stream of one mode relative to the data stream of another mode is greater than or equal to a threshold. For example, users can define warning1, warning2, warning3, warning4, warning5, and warning6 to respectively indicate that the delay between the data streams of the three modes is greater than or equal to a threshold, wherein warning1 indicates that the delay of the data stream of mode 1 relative to the data stream of mode 2 is greater than or equal to the threshold, warning2 indicates that the delay of the data stream of mode 1 relative to the data stream of mode 3 is greater than or equal to the threshold, warning3 indicates that the delay of the data stream of mode 2 relative to the data stream of mode 1 is greater than or equal to the threshold, warning4 indicates that the delay of the data stream of mode 2 relative to the data stream of mode 3 is greater than or equal to the threshold, warning5 indicates that the delay of the data stream of mode 3 relative to the data stream of mode 1 is greater than or equal to the threshold, and warning6 indicates that the delay of the data stream of mode 3 relative to the data stream of mode 2 is greater than or equal to the threshold.

[0193] The example can take an array to provide enumeration values, as shown below:

[0194] First Information

[0195] Enum value sequence{warning1, warning6}

[0196] }

[0197] In a tenth implementation manner, the first information includes an enumeration value and a delay of the first data stream relative to the second data stream, wherein the enumeration value is used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold.

[0198] This example can use an array to provide enumeration values ​​and delays, as shown below:

[0199] First Information

[0200] Enum value sequence{warning1, warning6}

[0201] Delay sequence {t1 delay , t2 delay}

[0202] }

[0203] Among them, warning1 and warning6 are enumeration values, t1 delay , t2 delay The above enumeration values ​​have a corresponding relationship with the delay. In this example, warning1 and t1 delay Correspondingly, the delay of the data flow of mode 1 relative to the data flow of mode 2 is t1 delay , warning6 and t2 delay Correspondingly, the delay of the data stream of mode 3 relative to the data stream of mode 2 is t2 delay .

[0204] This example uses two arrays to provide enumeration values ​​and delays respectively. In actual applications, you can also use one array to provide enumeration values ​​and delays, as shown below:

[0205] First Information

[0206] Enumeration value and delay sequence {warning1, warning6, t1 delay , t2 delay}

[0207] }

[0208] The first half of the sequence represents the enumeration value, and the second half of the sequence represents the delay corresponding to each enumeration value. In actual application, you can also set the delay corresponding to each enumeration value after the enumeration value, as shown below:

[0209] First Information

[0210] Enumeration value and delay sequence {warning1, t1 delay , warning6, t2 delay}

[0211] }

[0212] It should be noted that the above implementation methods are only some exemplary implementation methods of the present application. In other possible implementation methods of the embodiments of the present application, the delay in the first information can also be replaced by at least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, and the rate difference between the first data stream and the second data stream. Among them, the buffer difference between the first data stream and the second data stream can be the difference between the buffer size of the first data stream and the buffer size of the second data stream, and the rate difference between the first data stream and the second data stream can be the difference between the rate of the first data stream and the rate of the second data stream. By providing the above information, richer information can be provided for the network side to configure the resources for transmitting the first data stream or the second data stream, thereby achieving better results.

[0213] The above-mentioned first information can be encapsulated in a message or other information or structure for transmission. For example, the terminal can send user equipment assistant information (UAI) to the network device, and the UAI includes the above-mentioned first information. For another example, the terminal can send a medium access control layer control element (MAC CE) to the network device, and the MAC CE includes the first information. Alternatively, the terminal can send a PDCP layer control message to the network device, such as a PDCP layer control PDU, and the PDCP layer control message includes the first information.

[0214] Specifically, UAI includes a flow information structure (denoted as trafficInfo), which includes synchronization information (denoted as syncInfo) for multimodal services. The synchronization information is an array that includes the identifier of the first data flow (such as QFI 1) and the identifier of the second data flow (such as QFI 2). Furthermore, the array may also include the delay of the first data flow relative to the second data flow.

[0215] It should be noted that when syncInfo is represented as an array, it can include information about multiple data streams. This is explained below with an example:

[0216] Wherein, qfi1 and qfi2 are identifiers of the first data stream, and delay is the delay of the first data stream relative to the second data stream. It should be noted that other unexplained or undefined fields can be referred to the relevant content description of TS38.331 and will not be repeated here.

[0217] When the first information is sent in a MAC CE manner, the MAC CE may include an identifier of the first logical channel (recorded as LCID 1) and an identifier of the second logical channel (recorded as LCID 2). LCID 1 and LCID 2 may be 6 bits.

[0218] Figures 5, 6, and 7 illustrate various MAC CE formats. As shown in Figure 5, when the LCID is greater than 34, the MAC CE carries LCID 1 and LCID 2 in two bytes. LCID 1 and LCID 2 occupy a total of 12 bits, with the remaining four bits reserved. As shown in Figure 6, when the LCID is 33, the MAC CE adds two additional bytes to pad the extended LCID (eLCID). As shown in Figure 7, when the LCID is 34, the MAC CE adds one additional byte to pad the eLCID.

[0219] Furthermore, the MAC CE can be padded with additional delay, for example, by adding an additional byte. The delay unit can be milliseconds, with a maximum length of 255ms or 256ms. If the delay starts at 0, the maximum length can be 255ms; if the delay starts at 1, the maximum length can be 256ms.

[0220] When a PDCP layer control message is used to send the first information, an enumeration value of the type field may be added, and the added enumeration value is used to indicate synchronization information. As shown in Figure 8, the PDCP layer control message includes a type field and an identification field. The type field value is an enumeration value indicating synchronization information. In this example, the PDU Type field value may be 011, and the identification field is filled with the identifier of the first DRB (DRB_ID 1) and the identifier of the second DRB (DRB_ID 2).

[0221] Similar to MAC CE, PDCP layer control messages can also be padded with delay, for example, by adding an additional byte for delay padding. The delay unit can be milliseconds, with a maximum length of 255ms.

[0222] S306: The terminal receives the second information sent by the network device.

[0223] The second information may include resource configuration information for the first data stream and / or the second data stream. The terminal may receive the resource configuration information sent by the network device, or may not receive the resource configuration information sent by the network device. Whether the terminal receives the resource configuration information sent by the network device is related to whether the network device performs resource configuration. After receiving the first information, the network device may configure resources for transmitting the first data stream and / or the second data stream, or may not configure resources for transmitting the first data stream and / or the second data stream. For example, when a multimodal service has low synchronization requirements, resources for transmitting the first data stream and / or the second data stream may not be configured. The purpose of configuring resources for transmitting the first data stream and / or the second data stream is to synchronize the first data stream and the second data stream. Whether the first data stream and the second data stream are synchronized may also be affected by other factors. Therefore, configuring resources for transmitting data streams may or may not achieve synchronization. It should be noted that configuring resources for transmitting data streams may also have other purposes, and this application does not limit this.

[0224] The following description uses an example in which a network device configures resources for transmitting a first data stream and / or a second data stream, and a terminal receives resource configuration information sent by the network device.

[0225] In some possible implementations, the network device can be configured at the granularity of data streams (such as QoS streams). Accordingly, the resource configuration information can be resource configuration information at the granularity of data streams. Specifically, the network device can configure the priority, priority bit rate (PBR) or bucket depth (Bucket Size Duration, BSD) of the logical channel used to transmit the first data stream based on the first information. Accordingly, the resource configuration information can include the priority, PBR or BSD of the logical channel (such as the first logical channel) used to transmit the first data stream. The resource configuration information can also include the identifier of the first logical channel. Alternatively, the network device can configure the priority, PBR or BSD of the logical channel used to transmit the second data stream based on the first information. Accordingly, the resource configuration information can include the priority, PBR or BSD of the logical channel used to transmit the second data stream. The resource configuration information can also include the identifier of the second logical channel.

[0226] For ease of understanding, this application provides an example for illustration. In this example, the priority of the first logical channel is 3, and the priority of the second logical channel is 2, where a smaller value of the priority indicates a higher priority. When the first information is obtained, the first information is used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to the threshold. The network device can configure the priority of the first logical channel to be 1, or configure the priority of the second logical channel to be 4.

[0227] Furthermore, the network device may also configure the priority, PBR, or BSD of the first logical channel, and the priority, PBR, or BSD of the second logical channel based on the first information. For example, the first information may not only indicate that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold, but may also indicate the delay of the first data stream relative to the second data stream. If the delay is much greater than the threshold, the network device may reconfigure the resources used to transmit the first data stream and the resources used to transmit the second data stream, for example, configuring the priority of the first logical channel to be high priority and configuring the priority of the second logical channel to be low priority.

[0228] In some other possible implementations, the network device may also be configured with multimodal services as the granularity, and accordingly, the resource configuration information may also be resource configuration information with multimodal service granularity. Specifically, the network device may reconfigure the uplink grant (UL grant) of the multimodal service based on the first information. The uplink grant, also known as uplink scheduling permission or uplink scheduling authorization, is a physical control information from the access network device, indicating that the terminal may transmit data through the resources specified by the UL grant after receiving the UL grant. When reconfiguring the uplink UL grant, the resources specified by the uplink grant may be added. Accordingly, the second information may be the UL grant.

[0229] Similar to the first information, the second information can be encapsulated in a message or other information or structure for transmission. For example, a terminal can receive an RRC message including the second information sent by a network device. The RRC message including the second information can include a resource reconfiguration message. In some examples, the resource reconfiguration message can include logical channel configuration information, denoted as RRCReconfig(LogicChannelConfig). In some examples, the logical channel configuration information can include logical channel parameters, such as the priority of the logical channel.

[0230] S308. The terminal determines resources for transmitting the first data stream and / or resources for the second data stream according to the second information.

[0231] Specifically, the second information may include the priority of the first logical channel and / or the priority of the second logical channel. The terminal may allocate resources to the data stream from the resources specified by the UL grant in order of priority. The following example illustrates this: the priority of the first logical channel is 3, and the priority of the second logical channel is 2. When the terminal receives the second information, and the priority of the first logical channel in the second information is changed to 1, the terminal may prioritize allocating resources to the first data stream from the resources specified by the UL grant. In this way, the terminal can determine the resources used to transmit the first data stream and / or the resources for the second data stream.

[0232] The second information is resource configuration information at the granularity of the multimodal service. For example, in the case of a UL grant, the terminal can determine resources for the multimodal service based on the UL grant. The terminal can allocate resources to the multiple data streams from the resources specified in the UL grant based on the priority of the logical channels used to transmit the multiple data streams in the multimodal service. In this way, the terminal can determine the resources used to transmit the first data stream and / or the resources for the second data stream.

[0233] Taking into account that the capabilities of different terminals may differ, the terminal may also send capability information to the network device, and the capability information may be a capability related to synchronization. The capability information may indicate the terminal's ability to support data stream synchronization in multimodal services. The terminal's ability to support data stream synchronization in multimodal services may include supporting data stream synchronization in multimodal services, or not supporting data stream synchronization in multimodal services. Accordingly, the network device may send a second information to a terminal that supports data stream synchronization in multimodal services, to instruct the terminal to determine the resources for transmitting the first data stream and / or the second data stream based on the second information. For terminals that do not support data stream synchronization in multimodal services, the network device may use the original method for resource scheduling and data transmission. This can improve overall compatibility.

[0234] Based on the above description, the present application provides a data synchronization method. In this method, a terminal can monitor the latency of multiple data streams in a multimodal service, for example, the latency of one data stream relative to another. When the latency is greater than or equal to a threshold, the terminal can notify the network device to configure the resources of the data stream to ensure the synchronization of different data streams, meet service requirements, and improve the user experience.

[0235] The above describes the data synchronization method of the present application from the perspective of the terminal. The following describes the synchronization method of the present application from the perspective of the network device.

[0236] Referring to the flowchart of a data synchronization method shown in FIG9 , the method includes the following steps:

[0237] S902: The network device receives first information from the terminal.

[0238] The network device may be an access network device, such as a base station. The first information is used to indicate that the delay of the first data stream relative to the second data stream in the multimodal service is greater than or equal to a threshold. The threshold may be a threshold corresponding to the service type of the first data stream and the service type of the second data stream, such as a threshold of the first service type relative to the second service type. The synchronization requirements of data streams of different service types may be different, and accordingly, the thresholds corresponding to data streams of different service types (also referred to as synchronization delay thresholds, synchronization thresholds) may be different. The threshold may be configured by the network device, or predefined by a protocol, and accordingly, the threshold may be a predefined value.

[0239] The first information may include multiple implementations, which are described below.

[0240] In some possible implementations, the first information may include an identifier of the first data stream and an identifier of the second data stream. Furthermore, the first information may also include a time delay of the first data stream relative to the second data stream.

[0241] In some possible implementations, the first information may also include an identifier of a first logical channel and an identifier of a second logical channel. The first logical channel is used to transmit the first data stream, and the second logical channel is used to transmit the second data stream. Furthermore, the first information may also include a delay of the first data stream relative to the second data stream.

[0242] In some possible implementations, the first information may also include an identifier of a first DRB and an identifier of a second DRB. The first DRB is used to transmit the first data stream, and the second DRB is used to transmit the second data stream. Furthermore, the first information may also include a delay of the first data stream relative to the second data stream.

[0243] In some possible implementations, the first information may also include the mode of the first data stream and the mode of the second data stream. Furthermore, the first information may also include a time delay of the first data stream relative to the second data stream.

[0244] In some possible implementations, the first information may also include an enumeration value indicating that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold. Further, the first information may also include the delay of the first data stream relative to the second data stream.

[0245] The above implementations are only some exemplary implementations of the present application. In other possible implementations of the embodiments of the present application, the delay in the first information can also be replaced by at least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, and the rate difference between the first data stream and the second data stream. Among them, the buffer difference between the first data stream and the second data stream can be the difference between the buffer size of the first data stream and the buffer size of the second data stream, and the rate difference between the first data stream and the second data stream can be the difference between the rate of the first data stream and the rate of the second data stream. The network side obtains the above information and can more accurately configure the resources used to transmit the first data stream and / or the second data stream.

[0246] In specific implementation, the network device may obtain the first information in different ways, which are described below.

[0247] For example, the network device can obtain (for example, receive from the terminal) user equipment auxiliary information UAI, which includes the first information. Accordingly, the network device can parse the UAI to obtain the first information. For example, the UAI can carry the identifier of the first data stream and the identifier of the second data stream through trafficInfo. Furthermore, the UAI can also carry the delay through trafficInfo. Accordingly, the network device can parse the UAI, for example, parse the trafficInfo of the UAI, to obtain the identifier of the first data stream and the identifier of the second data stream. If trafficInfo also carries the delay of the first data stream relative to the second data stream, the above delay can also be obtained by parsing the UAI.

[0248] For another example, the network device may obtain (e.g., receive from a terminal) a MAC CE, which includes the first information. Accordingly, the network device may parse the MAC CE to obtain the first information. For example, the MAC CE may include an identifier of a first logical channel and an identifier of a second logical channel. The network device may parse the MAC CE and read the contents of the corresponding fields to obtain the identifier of the first logical channel and the identifier of the second logical channel. If the MAC CE also includes a delay of the first data stream relative to the second data stream, the delay may also be obtained by parsing the MAC CE.

[0249] For another example, the network device may obtain (for example, receive from a terminal) a PDCP layer control message, and the PDCP layer control message includes the first information. Accordingly, the network device may parse the above-mentioned PDCP layer control message to obtain the first information. For example, the PDCP control PDU may include an identifier of a first DRB and an identifier of a second DRB. The network device may parse the PDCP control PDU and read the contents of the corresponding fields to obtain the identifier of the first DRB and the identifier of the second DRB. If the PDCP control PDU also includes the delay of the first data stream relative to the second data stream, the above-mentioned delay may also be obtained by parsing the PDCP control PDU.

[0250] S904: The network device sends second information to the terminal.

[0251] The second information includes resource configuration information for the first data stream and / or the second data stream. When the network device receives the first information, it can configure resources for transmitting the first data stream and / or the second data stream, obtain resource configuration information for the first data stream and / or the second data stream, or not configure resources for transmitting the first data stream and / or the second data stream. For example, when the multimodal service has low synchronization requirements, resources for transmitting the first data stream and / or the second data stream may not be configured. The purpose of configuring resources for transmitting the first data stream and / or the second data stream is to synchronize the first data stream and the second data stream. Whether the first data stream and the second data stream are synchronized may also be affected by other factors. Therefore, configuring resources for transmitting data streams may or may not achieve synchronization. It should be noted that configuring resources for transmitting data streams may also be for other purposes, and this application does not limit this.

[0252] In some possible implementations, the network device can be configured at the data stream granularity. Accordingly, the resource configuration information can be resource configuration information at the data stream granularity. Specifically, the network device can configure the priority, PBR, or bucket depth (BSD) of the logical channel used to transmit the first data stream based on the first information. Accordingly, the resource configuration information can include the priority, PBR, or BSD used to transmit the first logical channel. The resource configuration information can also include the identifier of the first logical channel. Alternatively, the network device can configure the priority, PBR, or BSD of the logical channel used to transmit the second data stream based on the first information. Accordingly, the resource configuration information can include the priority, PBR, or BSD of the logical channel used to transmit the second data stream. The resource configuration information can also include the identifier of the second logical channel.

[0253] In other possible implementations, the network device may also be configured at the granularity of a multimodal service. Accordingly, the resource configuration information may also be resource configuration information at the granularity of a multimodal service. Specifically, the network device may reconfigure the UL grant for the multimodal service based on the first information. When reconfiguring the uplink UL grant, the resources specified by the uplink grant may be increased. Accordingly, the second information may be the UL grant.

[0254] When sending the second information, the network device may encapsulate the second information in a message or other information or structure for transmission. Specifically, the network device may encapsulate the second information in an RRC message. The RRC message may be a resource reconfiguration message. In some examples, the resource reconfiguration message may include logical channel configuration information, denoted as RRCReconfig(LogicChannelConfig). The logical channel configuration information may include logical channel parameters, such as the priority of the logical channel.

[0255] Based on the above description, the present application provides a data synchronization method. In this method, a network device can configure resources used to transmit the first data stream and / or the second data stream based on first information sent by a terminal indicating a timeout of the delay of the first data stream relative to the second data stream in a multimodal service, thereby ensuring the synchronization of data streams in different modes, meeting service requirements, and improving user experience.

[0256] Next, in combination with specific application scenarios, the data synchronization method of the present application is introduced from the perspective of interaction between the terminal and the network device.

[0257] 10 shows a schematic diagram of an application scenario of a data synchronization method, which specifically includes the following steps:

[0258] S1002: The terminal obtains a delay of a first data stream relative to a second data stream in a multimodal service. If the delay is greater than or equal to a threshold, S1004 is executed.

[0259] The first data stream and the second data stream can be data streams of different modes in a multimodal service. For ease of understanding, the embodiment of the present application is illustrated by an example in which the first data stream is a video type data stream and the second data stream is a posture and control type data stream. The terminal can obtain the first SDU (or PDU) in the PDCP layer video buffer and the first SDU (or PDU) in the posture and control buffer, extract the timestamp from the RTP header of the SDU, and determine the delay of the first data stream relative to the second data stream based on the timestamp extracted from the RTP header of the first SDU of different buffers. When the delay is greater than or equal to the threshold corresponding to the corresponding service type, the operation of S1004 can be triggered.

[0260] S1004. The terminal sends user equipment auxiliary information UAI including the first information to the network device.

[0261] Specifically, the terminal can send first information to the network device through the UAI architecture at the RRC layer. The first information is used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold. Furthermore, in addition to including the identifier of the first data stream and the identifier of the second data stream, the UAI may also include at least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, and the rate difference between the first data stream and the second data stream.

[0262] S1006. The network device configures resources for transmitting the first data stream and / or the second data stream according to the first information, and obtains second information.

[0263] The second information includes resource configuration information for the first data stream and / or the second data stream. Specifically, based on the first information, the network device can configure resources for transmitting the first data stream and / or the second data stream at a data stream granularity or a multimodal service granularity to obtain the resource configuration information. The specific configuration process can be found in the description of the embodiment in FIG9 and will not be repeated here.

[0264] S1008. The network device sends second information to the terminal.

[0265] The network device may encapsulate the second information in a message or other information or structure for transmission. Specifically, the network device may encapsulate the second information in an RRC message. The RRC message may be a resource reconfiguration message. In some examples, the resource reconfiguration message may include logical channel configuration information, denoted as RRCReconfig(LogicChannelConfig). The logical channel configuration information may include logical channel parameters, such as the priority of the logical channel.

[0266] S1010. The terminal determines, based on the second information, resources for transmitting the data stream of the first modality and / or the data stream of the second modality.

[0267] In some possible implementations, the second information may include the priority of the first logical channel and / or the priority of the second logical channel, and the terminal may allocate resources to the data stream from the resources specified by the UL grant in order of priority.

[0268] In other possible implementations, the second information is resource configuration information at the granularity of a multimodal service. For example, in the case of a UL grant, the terminal can determine resources for the multimodal service based on the UL grant. The terminal can then allocate resources to the multiple data streams from the resources specified in the UL grant based on the priorities of the logical channels used to transmit the multiple data streams in the multimodal service. In this way, the terminal can determine the resources used to transmit the first data stream and / or the resources for the second data stream.

[0269] FIG10 illustrates an example in which a terminal sends the first information through an RRC message to implement data synchronization. The following illustrates an example in which a terminal sends the first information through a MAC CE to implement data synchronization.

[0270] 11 shows a schematic diagram of an application scenario of a data synchronization method, which specifically includes the following steps:

[0271] S1102: The terminal obtains a delay of a first data stream relative to a second data stream in a multimodal service. If the delay is greater than or equal to a threshold, S1104 is executed.

[0272] The first data stream and the second data stream may be data streams of different modes in a multimodal service. The specific implementation of the terminal obtaining the time delay of the first data stream relative to the second data stream can be referred to the description of S1002, which will not be repeated here.

[0273] S1104. The terminal sends a MAC CE including first indication information to the network device.

[0274] Specifically, the terminal can send the first information to the network device through the MAC CE at the MAC layer. The first information includes an identifier of the first logical channel and an identifier of the second logical channel, which is used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold. Furthermore, in addition to the identifier of the first logical channel and the identifier of the second logical channel, the MAC CE may also include at least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, and the rate difference between the first data stream and the second data stream.

[0275] S1106. The network device configures resources for transmitting the first data stream and / or the second data stream according to the first information.

[0276] S1108. The network device sends second information to the terminal.

[0277] S1110. The terminal determines resources for transmitting the first data stream and / or the second data stream according to the second information.

[0278] The main difference between the embodiment shown in Figure 11 and the embodiment shown in Figure 10 is S1104. The specific implementation of other steps can refer to the relevant content description of the embodiment shown in Figure 10. For example, the specific implementation of S1106 to S1110 can refer to the relevant content description of S1006 to S1010 in the embodiment of Figure 10.

[0279] It should be noted that the relevant contents of the various embodiments of the present application can be related and referenced to each other.

[0280] Figure 12 is an example of the composition of an electronic device provided in an embodiment of the present application. The electronic device may be a first device, including but not limited to a base station and a core network unit. Figure 12 shows a simplified schematic diagram of the base station structure. The base station includes parts 1210, 1220, and 1230. Part 1210 is mainly used for baseband processing, controlling the base station, etc.; Part 1210 is usually the control center of the base station, which can usually be called a processor, and is used to control the base station to perform the processing operations on the first device side in the above method embodiment. Part 1220 is mainly used to store computer program code and data. Part 1230 is mainly used for receiving and transmitting radio frequency signals and converting radio frequency signals into baseband signals; Part 1230 can usually be called a transceiver module, a transceiver, a transceiver circuit, or a transceiver, etc. The transceiver module of part 1230 can also be called a transceiver or a transceiver, etc., which includes an antenna 1233 and a radio frequency circuit (not shown in the figure), wherein the radio frequency circuit is mainly used for radio frequency processing. Alternatively, the device for implementing the receiving function in section 1230 may be considered a receiver, and the device for implementing the transmitting function may be considered a transmitter, that is, section 1230 includes a receiver 1232 and a transmitter 1231. A receiver may also be referred to as a receiving module, a receiver, or a receiving circuit, and a transmitter may be referred to as a transmitting module, a transmitter, or a transmitting circuit.

[0281] Sections 1210 and 1220 may include one or more boards, each of which may include one or more processors and one or more memories. The processor is used to read and execute programs in the memory to implement baseband processing functions and control the base station. If multiple boards are present, the boards may be interconnected to enhance processing capabilities. As an optional implementation, multiple boards may share one or more processors, multiple boards may share one or more memories, or multiple boards may simultaneously share one or more processors.

[0282] For example, in one implementation, the transceiver module in section 1230 is used to execute the transceiver-related processes executed by the base station (first device) in the aforementioned method embodiment. The processor in section 1210 is used to execute the processing-related processes executed by the base station in the aforementioned method embodiment.

[0283] It should be understood that FIG12 is merely an example and not a limitation, and the network device including the processor, memory, and transceiver may not rely on the structure shown in FIG12 .

[0284] Figure 13 is an example of the composition of another electronic device provided in an embodiment of the present application. The electronic device can be a second device, which can be a terminal, including but not limited to mobile phones, smart wearable devices (such as smart watches), and other electronic devices. Taking a mobile phone as an example, the electronic device can include a processor 310, an external memory interface 320, an internal memory 321, a display 330, a camera 340, an antenna 1, an antenna 2, a mobile communication module 350, and a wireless communication module 360, etc.

[0285] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown, or some components may be combined or separated, or the components may be arranged differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0286] The processor 310 may include one or more processing units. For example, the processor 310 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0287] It is understood that the interface connection relationship between the modules illustrated in this embodiment is only a schematic illustration and does not constitute a structural limitation of the electronic device. In other embodiments of the present application, the electronic device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0288] External memory interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with processor 310 via external memory interface 320 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0289] The internal memory 321 can be used to store computer executable program code, and the executable program code includes instructions. The processor 310 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 321. The internal memory 321 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the internal memory 321 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 310 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 321, and / or the instructions stored in the memory provided in the processor.

[0290] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, modem processor and baseband processor.

[0291] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in an electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0292] The mobile communication module 350 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to electronic devices. The mobile communication module 350 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 350 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 350 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 350 can be set in the processor 310. In some embodiments, at least some of the functional modules of the mobile communication module 350 can be set in the same device as at least some of the modules of the processor 310.

[0293] In some embodiments, the electronic device initiates or receives a call request via the mobile communication module 350 and the antenna 1 .

[0294] Furthermore, an operating system runs on the aforementioned components, such as the iOS operating system, the Android operating system, and the Windows operating system. Application programs can be installed and run on the operating system. Those skilled in the art will clearly understand that, for ease of description and brevity, the explanation and beneficial effects of any of the aforementioned electronic devices can be referred to the corresponding method embodiments provided above, and will not be further elaborated here.

[0295] The present application also provides a communication system, which may include a first device as shown in FIG12 (for example, a network device such as a base station) and a second device as shown in FIG13 (for example, a terminal such as a mobile phone).

[0296] In this application, a terminal or network device may include a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer may include hardware such as a central processing unit (CPU), a memory management unit (MMU), and memory (also known as main memory). The operating system of the operating system layer may be any one or more computer operating systems that implement business processing through processes, such as the Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system. The application layer may include applications such as browsers, address books, word processing software, and instant messaging software.

[0297] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices and modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0298] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or module, which can be electrical, mechanical or other forms.

[0299] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected to achieve the purpose of the present embodiment according to actual needs.

[0300] In addition, the functional modules in the various embodiments of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The above-mentioned integrated modules may be implemented in the form of hardware or software functional modules.

[0301] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the part that essentially contributes to the technical solution of the present application or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the process of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.

[0302] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data synchronization method, characterized in that, The method includes: Obtaining the latency of the first data stream relative to the second data stream in the multi-modal service; When the latency of the first data stream relative to the second data stream is greater than or equal to a threshold, sending first information to a network device, where the first information is used to indicate that the latency of the first data stream relative to the second data stream is greater than or equal to the threshold; Receiving second information sent by the network device, where the second information includes resource configuration information for the first data stream and / or the second data stream; Determining resources for transmitting the first data stream and / or resources for the second data stream according to the second information.

2. The method according to claim 1, characterized in that The first information includes: The identifier of the first data stream and the identifier of the second data stream; or, At least one of the latency of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the identifier of the first data stream, the identifier of the second data stream; or, The identifier of the first logical channel and the identifier of the second logical channel, where the first logical channel is used to transmit the first data stream and the second logical channel is used to transmit the second data stream; or, At least one of the latency of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the identifier of the first logical channel, the identifier of the second logical channel; or, The identifier of the first data radio bearer (DRB) and the identifier of the second DRB, where the first DRB is used to transmit the first data stream and the second DRB is used to transmit the second data stream; or, At least one of the latency of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the identifier of the first DRB, the identifier of the second DRB; or, The modality of the first data stream and the modality of the second data stream; or, At least one of the latency of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the modality of the first data stream, the modality of the second data stream; or, An enumerated value, where the enumerated value is used to indicate that the latency of the first data stream relative to the second data stream is greater than or equal to the threshold; or, At least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and an enumerated value, where the enumerated value is used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold.

3. The method according to claim 1 or 2, characterized in that, Sending the first information to the network device includes: Sending user equipment assistance information (UAI) to the network device, where the UAI includes the first information; or, Sending a media access control layer control element (MAC CE) to the network device, where the MAC CE includes the first information; or, Sending a packet data convergence protocol (PDCP) layer control message to the network device, where the PDCP layer control message includes the first information.

4. The method according to claim 3, wherein The UAI includes a stream information structure, and the stream information structure includes synchronization information for the multimodal service, and the synchronization information is an array including the identifier of the first data stream and the identifier of the second data stream.

5. The method according to claim 4, wherein The array further includes at least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, and the rate difference between the first data stream and the second data stream.

6. The method according to claim 3, characterized in that The MAC CE includes the identifier of the first logical channel and the identifier of the second logical channel.

7. The method according to claim 3, wherein The PDCP layer control message includes a type field and an identifier field. The type field is used to indicate the enumerated value of the synchronization information, and the identifier field is used to indicate the identifier of the first data radio bearer (DRB) and the identifier of the second DRB.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: Sending the capability information of the terminal to the network device, where the capability information is used to indicate the support capability of the terminal for data stream synchronization in the multimodal service.

9. The method according to any one of claims 1 to 8, characterized in that, The threshold is configured by the network device, or the threshold is a predefined value.

10. The method according to any one of claims 1 to 9, characterized in that The method further includes: Receiving configuration information sent by the network device, where the configuration information is used to indicate the data streams to be synchronized, and the data streams to be synchronized include the first data stream and the second data stream.

11. The method according to any one of claims 1 to 10, characterized in that, The method is executed by a terminal, and the terminal is connected to multiple sensing devices, and each of the multiple sensing devices is used to collect at least one data stream of the multimodal service.

12. A data synchronization method, characterized in that, The method includes: Receiving first information from the terminal, where the first information is used to indicate that the delay of the first data stream relative to the second data stream in the multimodal service is greater than or equal to a threshold; Sending second information to the terminal, where the second information includes resource configuration information for the first data stream and / or the second data stream.

13. The method according to claim 12, characterized in that, The method further includes: Configuring resources for transmitting the first data stream and / or the second data stream, and obtaining the resource configuration information for the first data stream and / or the second data stream.

14. The method according to claim 12 or 13, characterized in that, The first information includes: The identifier of the first data stream and the identifier of the second data stream; or, At least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the identifier of the first data stream, the identifier of the second data stream; or, The identifier of the first logical channel and the identifier of the second logical channel, where the first logical channel is used to transmit the first data stream and the second logical channel is used to transmit the second data stream; or, At least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the identifier of the first logical channel, the identifier of the second logical channel; or, The identifier of the first data radio bearer (DRB) and the identifier of the second DRB, where the first DRB is used to transmit the first data stream and the second DRB is used to transmit the second data stream; or, At least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the identifier of the first DRB, the identifier of the second DRB; or, The mode of the first data stream and the mode of the second data stream; or, At least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the mode of the first data stream, the mode of the second data stream; or, An enumerated value used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold; or, At least one of the delay of the first data stream relative to the second data stream, the buffer size of the first data stream, the buffer size of the second data stream, the buffer difference between the first data stream and the second data stream, the rate of the first data stream, the rate of the second data stream, the rate difference between the first data stream and the second data stream, and the enumerated value used to indicate that the delay of the first data stream relative to the second data stream is greater than or equal to a threshold.

15. The method according to any one of claims 12 to 14, characterized in that, Receiving the first information from the terminal includes: Receiving user equipment assistance information (UAI) from the terminal, where the UAI includes the first information; or, Receive a Media Access Control layer control element MAC CE from a terminal, where the MAC CE includes the first information; or, Receive a Packet Data Convergence Protocol PDCP layer control message from a terminal, where the PDCP layer control message includes the first information.

16. An electronic device, characterized in that, The electronic device includes: A memory for storing computer programs or computer instructions; A processor for executing the computer programs or computer instructions stored in the memory, so that the electronic device executes the method according to any one of claims 1 to 11.

17. An electronic device, characterized in that, The electronic device includes: A memory for storing computer programs or computer instructions; A processor for executing the computer programs or computer instructions stored in the memory, so that the electronic device executes the method according to any one of claims 12 to 15.

18. A communication system, characterized in that, The system includes a first device and the second device, the first device is configured to execute the method according to any one of claims 1 to 11, and the second device is configured to execute the method according to any one of claims 12 to 15.

19. A computer storage medium for storing a computer program, which when executed is used to implement the method according to any one of claims 1 to 15.

Images