Communication method and communication apparatus

Abstract

Embodiments of the present application provide a communication method and a communication device. The method comprises: a terminal device sending first indication information to a first communication device, the first indication information being used to instruct the first communication device to render first foreground content, the first foreground content being foreground content to be displayed by the terminal device; and the terminal device receiving first information from the first communication device, the first information comprising second foreground content and second background content, wherein the second foreground content is obtained by rendering the first foreground content, and the second background content is determined based on first background content, the first background content being background content to be displayed by the terminal device. According to the present application, the first communication device determines the second foreground content and the second background content, and sends them to the terminal device. The terminal device combines the second background content and the second foreground content, and displays them, thereby reducing the computing power requirement of the terminal device, making the terminal device lightweight, and improving the user experience.

Description

Technical Field

[0001] This application relates to the field of communications, and more specifically, to a communication method and a communication device. Background Technology

[0002] Extended reality (XR) refers to an interactive environment that combines real and virtual elements, created through computer technology and wearable devices. XR is built upon augmented reality (AR), virtual reality (VR), and mixed reality (MR). In other words, to avoid confusion, XR is a general term encompassing AR, VR, and MR. The goal of XR services is to utilize high-speed networks and technologies such as 360-degree imaging to achieve an interactive and immersive experience.

[0003] Currently, there is limited research on XR, and improving the actual user experience for XR services is an urgent issue that needs to be addressed. Summary of the Invention

[0004] This application provides a communication method and a communication device that enable lightweight terminal devices and further improve the user's actual experience.

[0005] In a first aspect, a communication method is provided, which can be executed by a terminal device or by a chip or circuit disposed in the terminal device. This application does not limit the method in this regard. For ease of description, the following description takes execution by a terminal device as an example.

[0006] The method includes:

[0007] The terminal device sends first instruction information to the first communication device, the first instruction information being used to instruct the first communication device to render first foreground content, the first foreground content being the foreground content to be displayed by the terminal device; the terminal device receives first information from the first communication device, the first information including second foreground content and second background content, wherein the second foreground content is rendered based on the first foreground content, the second background content is decoded based on a third background content, the third background content is determined based on the first background content, and the first background content is the background content to be displayed by the terminal device.

[0008] According to the method provided in this application embodiment, a terminal device sends first instruction information to a first communication device, the first instruction information instructing the first communication device to render first foreground content, the first foreground content being the foreground content to be displayed by the terminal device. The terminal device receives first information from the first communication device, the first information including second background content and second foreground content. The terminal device merges the second background content and the second foreground content and displays them. The terminal device does not need to perform corresponding processing on the foreground and background content to be displayed; it only needs to receive the processed background and foreground content from the first communication device, reducing the computational requirements of the terminal device, thereby making the terminal device lightweight and improving the user experience.

[0009] It should be understood that the first background content in this application is the background content to be displayed by the terminal device, and the first foreground content is the foreground content to be displayed by the terminal device. Both the background content to be displayed and the foreground content to be displayed belong to the screen to be displayed by the terminal device (or the content to be displayed, or the screen to be processed, or the content to be processed), and both are screens or contents that have not been processed by the terminal device, the first communication device, or the second communication device.

[0010] It should be noted that the first communication device in this application may be a computing power node at the edge of the terminal device (or a nearby computing power node), or the first communication device may be a computing power node that provides services to the terminal device.

[0011] It should also be noted that the first communication device may be one or more computing nodes, and the terminal device may be one or more terminal devices; this application does not limit the scope of the application.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the terminal device sends user interaction information to the second communication device, the user interaction information being used to determine the first background content.

[0013] Based on the above scheme, after the terminal device determines the user interaction information, it sends the user interaction information to the second communication device. The user interaction information is used by the second communication device to further determine the first background content.

[0014] In conjunction with the first aspect, in some implementations of the first aspect, the third background content is obtained by the second communication device rendering and encoding based on the first background content.

[0015] Based on the above scheme, after the second communication device determines the first background content according to the user interaction information, it renders and encodes the first background content to obtain the third background content. The second communication device sends a second instruction to the first communication device, instructing the first communication device to decode the third background content, thereby avoiding the terminal device from decoding the third background content and reducing the complexity and computational requirements of the terminal device.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, the third background content is obtained by the second communication device from locally cached background content based on the first background content.

[0017] Based on the above scheme, after determining the first background content based on user interaction information, the second communication device queries the locally cached background content based on the first background content. In this process, caching frequently accessed content locally helps reduce end-to-end data transmission latency and improves system performance.

[0018] In conjunction with the first aspect, in some implementations of the first aspect, the terminal device determines the first indication information based on the user interaction information and the second information, wherein the second information is used to indicate the state of the first communication device, and the second information includes one or more of the following: the channel state of the first communication device, the total computing resources of the first communication device, the available computing resources of the first communication device, the remaining power of the first communication device, or the storage space of the first communication device.

[0019] Based on the above scheme, the terminal device determines the first indication information based on user interaction information and the second information. The terminal device further determines the first indication information based on the status of the first communication device. For example, when the first communication device has ample available computing resources, the computing demand indicated by the first indication information sent by the terminal device to the first communication device may be higher. When the remaining power of the first communication device is low, the computing demand indicated by the first indication information sent by the terminal device to the first communication device may be low or zero. The terminal device's further determination of the first indication information based on the status of the first communication device can reduce end-to-end transmission latency through distributed rendering, improve rendering quality, and enhance the user experience.

[0020] In conjunction with the first aspect, in some implementations of the first aspect, the user interaction information includes one or more of the following: changes in the user's posture on the terminal device, changes in the user's position on the terminal device, or user control commands on the terminal device.

[0021] It should be understood that changes in user posture, position, and control commands of a terminal device refer to information about the user using or controlling the terminal device.

[0022] Secondly, a communication method is provided. This method can be executed by a first communication device, or by a chip or circuit disposed in the first communication device. This application does not limit the method in this regard. For ease of description, the following description takes execution by the first communication device as an example.

[0023] The method includes:

[0024] A first communication device receives first instruction information from a terminal device, the first instruction information instructing the first communication device to render first foreground content, the first foreground content being the foreground content to be displayed by the terminal device; the first communication device receives second instruction information from a second communication device, the second instruction information instructing the first communication device to decode third background content, the third background content being determined based on the first background content, the first background content being the background content to be displayed by the terminal device; the first communication device sends first information to the terminal device, the first information including second foreground content and second background content, wherein the second foreground content is rendered based on the first foreground content, and the second background content is decoded based on the third background content.

[0025] According to the method provided in the embodiments of this application, a first communication device receives first instruction information and second instruction information. The first instruction information instructs the first communication device to render first foreground content, which is the foreground content to be displayed by the terminal device. The second instruction information instructs the first communication device to decode third background content, which is determined by the second communication device based on the first background content. The first communication device sends first information to the terminal device, which includes the second background content and the second foreground content obtained after processing by the first communication device. The terminal device merges the second background content and the second foreground content and displays them. The terminal device does not need to perform corresponding processing on the foreground and background content to be displayed; it only needs to receive the background and foreground content processed by the first communication device. This reduces the computational requirements of the terminal device, making the terminal device lightweight and improving the user experience.

[0026] In conjunction with the second aspect, in some implementations of the second aspect, the third background content is obtained by the second communication device rendering and encoding based on the first background content.

[0027] For example, when the second communication device is a cloud server, the second communication device renders and encodes the first background content to obtain the third background content.

[0028] In conjunction with the second aspect, in some implementations of the second aspect, the third background content is obtained by the second communication device from locally cached background content based on the first background content.

[0029] For example, when the second communication device is a core network device or an access network device, after determining the first background content based on user interaction information, the second communication device queries the locally cached background content based on the first background content. Cacheing this popular content locally helps reduce end-to-end data transmission latency and improves system performance.

[0030] In conjunction with the second aspect, in some implementations of the second aspect, the first communication device sends second information, the second information being used to indicate the state of the first communication device, wherein the second information includes one or more of the following: the channel state of the first communication device, the total computing resources of the first communication device, the computable resources of the first communication device, the remaining power of the first communication device, or the storage space of the first communication device.

[0031] Based on the above scheme, the first communication device sends second information, which indicates the status of the first communication device. The terminal device further determines the first indication information based on the user interaction information and the second information, thereby improving the quality of content rendering and enhancing the user experience.

[0032] Thirdly, a communication method is provided, which can be executed by a second communication device or by a chip or circuit disposed in the second communication device. This application does not limit the method in this regard. For ease of description, the following description takes execution by the second communication device as an example.

[0033] The method includes:

[0034] The second communication device receives user interaction information from the terminal device, the user interaction information being used to determine a first background content, the first background content being the background content to be displayed by the terminal device; the second communication device sends second instruction information to the first communication device, the second instruction information being used to instruct the first communication device to decode a third background content, the third background content being determined based on the first background content.

[0035] According to the method provided in the embodiments of this application, the second communication device determines the first background content to be displayed by the terminal device based on user interaction information, and sends the second instruction information to the first communication device, instructing the first communication device to decode the third background content. The third background content is determined based on the first background content, which avoids the terminal device decoding the third background content, reduces the computing requirements of the terminal device, thereby making the terminal device lightweight and improving the user's actual experience.

[0036] In conjunction with the third aspect, in some implementations of the third aspect, the third background content is obtained by the second communication device rendering and encoding based on the first background content.

[0037] For example, when the second communication device is a cloud server, the second communication device renders and encodes the first background content to obtain the third background content.

[0038] In conjunction with the third aspect, in some implementations of the third aspect, the third background content is obtained by the second communication device from locally cached background content based on the first background content.

[0039] For example, when the second communication device is a core network device or an access network device, after determining the first background content based on user interaction information, the second communication device queries the locally cached background content based on the first background content. Cacheing this popular content locally helps reduce end-to-end data transmission latency and improves system performance.

[0040] In conjunction with the third aspect, in some implementations of the third aspect, the second communication device sends the second indication information to the first communication device based on the second information, wherein the second information is used to indicate the status of the first communication device, and the second information includes one or more of the following: the channel status of the first communication device, the total computing resources of the first communication device, the available computing resources of the first communication device, the remaining power of the first communication device, or the storage space of the first communication device.

[0041] In conjunction with the third aspect, in some implementations of the third aspect, the second communication device sends the second indication information to the first communication device based on the second information and the third information, wherein the third information includes the bitrate identifier of the third background content.

[0042] In conjunction with the third aspect, in some implementations of the third aspect, the third information also includes block identifiers for the third background content.

[0043] In conjunction with the third aspect, in some implementations of the third aspect, the user interaction information includes one or more of the following: changes in the user's posture on the terminal device, changes in the user's position on the terminal device, or user control commands on the terminal device.

[0044] Fourthly, a communication device is provided, comprising: a transceiver unit configured to send first instruction information to a first communication device, the first instruction information being configured to instruct the first communication device to render first foreground content, the first foreground content being foreground content to be displayed by the terminal device; the transceiver unit further configured to receive first information from the first communication device, the first information including second foreground content and second background content, wherein the second foreground content is rendered based on the first foreground content, the second background content is decoded based on third background content, the third background content is determined based on the first background content, and the first background content is background content to be displayed by the terminal device.

[0045] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the transceiver unit is further configured to send user interaction information to the second communication device, the user interaction information being used to determine the first background content.

[0046] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the third background content is obtained by the second communication device rendering and encoding based on the first background content.

[0047] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the third background content is obtained by the second communication device from locally cached background content based on the first background content.

[0048] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the processing unit is configured to determine the first indication information based on the user interaction information and the second information, wherein the second information is used to indicate the state of the first communication device, and the second information includes one or more of the following: the channel state of the first communication device, the total computing resources of the first communication device, the available computing resources of the first communication device, the remaining power of the first communication device, or the storage space of the first communication device.

[0049] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the user interaction information includes one or more of the following: changes in the user's posture on the terminal device, changes in the user's position on the terminal device, or user control commands on the terminal device.

[0050] Fifthly, a communication device is provided, comprising: a transceiver unit configured to receive first indication information from a terminal device, the first indication information being configured to instruct a processing unit to render first foreground content, the first foreground content being foreground content to be displayed by the terminal device; the transceiver unit configured to receive second indication information from a second communication device, the second indication information being configured to instruct a processing unit to decode third background content, the third background content being determined based on first background content, the first background content being background content to be displayed by the terminal device; the transceiver unit further configured to send first information to the terminal device, the first information including second foreground content and second background content, wherein the second foreground content is rendered based on the first foreground content, and the second background content is decoded based on the third background content.

[0051] In conjunction with the fifth aspect, in some implementations of the fifth aspect, the third background content is obtained by the second communication device rendering and encoding the first background content.

[0052] In conjunction with the fifth aspect, in some implementations of the fifth aspect, the third background content is obtained by the second communication device from locally cached background content based on the first background content.

[0053] In conjunction with the fifth aspect, in some implementations of the fifth aspect, the transceiver unit is further configured to transmit second information, the second information being used to indicate the state of the first communication device, wherein the second information includes one or more of the following: the channel state of the first communication device, the total computing resources of the first communication device, the computable resources of the first communication device, the remaining power of the first communication device, or the storage space of the first communication device.

[0054] In a sixth aspect, a communication device is provided, comprising: a transceiver unit configured to receive user interaction information from a terminal device, the user interaction information being used to determine first background content, the first background content being background content to be displayed by the terminal device; the transceiver unit is further configured to send second instruction information to a first communication device, the second instruction information being used to instruct the first communication device to decode third background content, the third background content being determined based on the first background content.

[0055] In conjunction with the sixth aspect, in some implementations of the sixth aspect, the processing unit determines the first background content based on the user interaction information; the processing unit determines the third background content based on the first background content.

[0056] In conjunction with the sixth aspect, in some implementations of the sixth aspect, the processing unit is used to determine the third background content based on the rendering and encoding of the first background content.

[0057] In conjunction with the sixth aspect, in some implementations of the sixth aspect, the processing unit is configured to determine the third background content from locally cached background content based on the first background content.

[0058] In conjunction with the sixth aspect, in some implementations of the sixth aspect, the transceiver unit is further configured to send the second indication information to the first communication device according to the second information, wherein the second information is used to indicate the status of the first communication device, and the second information includes one or more of the following: the channel status of the first communication device, the total computing resources of the first communication device, the available computing resources of the first communication device, the remaining power of the first communication device, or the storage space of the first communication device.

[0059] In conjunction with the sixth aspect, in some implementations of the sixth aspect, the processing unit is further configured to send the second indication information to the first communication device based on the second information and the third information, wherein the third information includes the bitrate identifier of the third background content.

[0060] In conjunction with the sixth aspect, in some implementations of the sixth aspect, the third information also includes the block identifier of the third background content.

[0061] In conjunction with the sixth aspect, in some implementations of the sixth aspect, the user interaction information includes one or more of the following: changes in the user's posture on the terminal device, changes in the user's position on the terminal device, or user control commands on the terminal device.

[0062] A seventh aspect provides a communication device comprising: a processor coupled to a memory for storing programs or instructions, wherein when the programs or instructions are executed by the processor, the communication device performs the methods of the first aspect, the second aspect, or the third aspect and various possible implementations thereof.

[0063] Optionally, there may be one or more processors and one or more memories.

[0064] Optionally, the memory can be integrated with the processor, or the memory can be a separate device from the processor.

[0065] Optionally, the relay device may also include a transmitter and a receiver.

[0066] Eighthly, a communication system is provided, comprising: a terminal device for performing the method of the first aspect and any possible implementation thereof; a first communication device for performing the method of the second aspect and any possible implementation thereof; and a second communication device for performing the method of the third aspect and any possible implementation thereof.

[0067] Ninth aspect, a computer-readable medium is provided that stores a computer program (also referred to as code or instructions) that, when executed, causes a computer to perform the methods of any possible implementation of the first, second, or third aspect described above.

[0068] In a tenth aspect, a chip system is provided, including a memory and a processor, the memory for storing a computer program, the processor for storing the computer program, and the processor for calling and running the computer program from the memory, causing a communication device equipped with the chip system to perform the methods of any of the first, second, or third aspects and their possible implementations described above.

[0069] The chip system may include an input chip or interface for transmitting information or data, and an output chip or interface for receiving information or data. Attached Figure Description

[0070] Figure 1 This is a schematic diagram of a distributed rendering architecture provided in this application.

[0071] Figure 2 This is a schematic diagram of another distributed rendering architecture provided in this application.

[0072] Figure 3 This is a schematic block diagram of a communication method provided in this application.

[0073] Figure 4 This is a schematic block diagram of a communication method provided in this application.

[0074] Figure 5 This is a schematic block diagram of a communication device provided in this application.

[0075] Figure 6 This is a schematic block diagram of another communication device provided in this application.

[0076] Figure 7 This is a schematic diagram of an example display screen provided in this application. Detailed Implementation

[0077] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0078] The technical solutions of this application embodiment can be applied to various communication systems, such as: Global System of Mobile communication (GSM) system, Code Division Multiple Access (CDMA) system, Wideband Code Division Multiple Access (WCDMA) system, General Packet Radio Service (GPRS), Long Term Evolution (LTE) system, LTE Frequency Division Duplex (FDD) system, LTE Time Division Duplex (TDD) system, Universal Mobile Telecommunication System (UMTS), Worldwide Interoperability for Microwave Access (WIMAX) communication system, future 5th Generation (5G) system, or New Radio (NR), etc.

[0079] The terminal device in this application embodiment is also referred to as user equipment (UE), mobile station (MS), mobile terminal (MT), VR glasses, edge computing node, etc. A terminal device refers to a device that provides voice and / or data connectivity to a user, and also includes devices capable of sidelink communication, such as vehicle-mounted terminals or handheld terminals capable of V2X communication. VR glasses need to complete tasks such as reporting user interaction information, foreground extraction, foreground content rendering task allocation, and bitstream merging.

[0080] The first communication device in this application embodiment can be an edge computing node of the terminal device, or a computing node capable of providing services to the terminal device. The computing node needs to complete channel state information (CSI) feedback, computing resource registration, computing status feedback, and assist the VR terminal device in completing decoding and rendering tasks. The computing node can be a PC (personal computer) or mobile phone in a home network, or a Customer Premise Equipment (CPE), or an Intelligence Network Gateway (ING), or a content delivery network (CDN) device, or other VR devices. Computing nodes can be connected via sidelink or 5GS. This application embodiment is not limited in this respect.

[0081] The second communication device in this application embodiment may be an access network device, a core network device, or a cloud server (also known as an application layer).

[0082] In this application's embodiments, the access network device refers to a radio access network (RAN) node (or device) that connects a terminal to a wireless network; it can also be called a base station, network device, or communication device. For example, the access network device can be an NR gNB, an LTE eNB, or various other types of base stations. The base station adopts a centralized unit-distributed unit (CU-DU) separation architecture.

[0083] The core network in this embodiment is mainly responsible for processing and routing app data. The user plane function (UPF) in the core network is responsible for connecting to external networks. It includes the functions of the LTE Serving Gateway (SGW) and Public Data Network Gateway (PDN-GW), and is mainly responsible for packet forwarding, billing information statistics, etc.

[0084] The application server (App) in this embodiment provides users with various application data. This application assumes that the server provides VR video service data.

[0085] Extended Reality (XR) in this application refers to various environments that combine reality and virtuality generated by computing technology and wearable devices, as well as human-computer interaction. Specifically, it includes the following typical forms: Augmented Reality (AR), Mixed Reality (MR), and Virtual Reality (VR).

[0086] To facilitate understanding of the communication method proposed in this application, this application selects VR services in XR as an example and provides a detailed description of the method provided. The method provided in this application can also be applied to other services in XR, such as AR and MR services. This application does not limit its application to these applications.

[0087] The complete process of a VR service from initiation to termination is as follows: The VR terminal device (e.g., a head-mounted display) first captures changes in the user's posture, position, and commands, and sends this user interaction information as a user request to the server. The server then renders the video frames associated with the user request and transmits the rendered image back to the VR terminal device for display. Rendering refers to the process of generating images from a model, where the model is a representation of a three-dimensional object or virtual environment defined by a programming language or data structure.

[0088] VR services require both low latency and high speed. VR video services require a motion-to-photo (MTP) latency of less than 25ms, meaning the time from a change in the user's posture or position to the appearance of the video image on the headset must be less than 25ms. If the MTP latency exceeds 25ms, users will experience dizziness. Furthermore, for 2K resolution and 60FPS VR video services, the bandwidth requirement is approximately 30Mbps. As VR services evolve, they are gradually moving from initial immersion to deep immersion, characterized by resolution improvements from 2K to 4K and 8K, frame rates from 60FPS to 90FPS and 120FPS, and bandwidth requirements increasing from Mbps to Gbps. Moreover, with the continuous evolution of VR services, the computing power required to render real-time VR video is increasing significantly. To meet the enormous computing power demands of VR services, they are gradually evolving from local rendering to cloud rendering. Therefore, this application aims to propose a novel distributed rendering architecture to meet the computing power requirements of VR rendering, thereby further reducing the transmission latency of VR services and enabling VR terminal devices to be extremely lightweight.

[0089] Currently, VR rendering architectures mainly include local rendering, cloud rendering, asynchronous edge-cloud rendering, and collaborative edge-cloud rendering. These rendering architectures will be introduced below:

[0090] 1. Local rendering

[0091] All rendering tasks are performed on the local VR terminal device or a server connected to the VR glasses via a wired connection. Local rendering, limited by the computing power of the terminal's graphics processing unit (GPU), can only achieve a resolution of 2K-4K and a frame rate of 30fps-60fps. The advantage of local rendering is its low latency, which can meet the user's real-time needs. However, its drawbacks are also significant. On the one hand, the computing power of the VR terminal device or local server is limited, only supporting the rendering of low-quality video; on the other hand, the VR terminal device needs to be connected to the server via a wired connection, thus limiting the mobility of users using the VR terminal device.

[0092] 2. Cloud rendering

[0093] VR terminal devices capture changes in the user's posture and position, uploading these changes as user interaction information to the cloud. The cloud then renders the corresponding video footage in real time based on this information, encodes the rendered footage, and transmits it back to the VR terminal device. The VR terminal device decodes the video footage and displays it on the head-mounted display. While cloud rendering architecture leverages the computing power of cloud clusters to provide high-definition VR video rendering services, end-to-end transmission latency has become a bottleneck limiting the further development of VR terminal device services towards deep immersion.

[0094] 3. Asynchronous rendering between edge and cloud

[0095] The core idea of ​​edge-cloud asynchronous rendering is that the VR server pre-renders viewpoint image frames and transmits them to the VR terminal device, which can then perform the latest posture correction based on this. Compared to cloud rendering architecture, the improvement of edge-cloud asynchronous rendering architecture lies in the introduction of asynchronous time warp technology. Asynchronous time warp technology means that if the current video frame fails to be transmitted to the VR terminal device within the MTP latency range, the VR terminal device can compensate for the deviation caused by head rotation by rotating the last successfully received video frame. Therefore, in edge-cloud asynchronous rendering architecture, the end-to-end transmission latency can be relaxed from 25ms in cloud rendering architecture to 50~70ms, enabling existing networks to support cloud VR video services. However, the application of asynchronous time warp technology also causes black borders or distortion in the video frames viewed by the user, thus impairing the user experience.

[0096] 4. Edge-Cloud Collaborative Rendering

[0097] To avoid black borders or distortion caused by asynchronous time warping techniques, edge-cloud collaborative rendering architecture has been widely discussed in academia, but it has not yet been commercialized due to application ecosystem issues. The core idea of ​​edge-cloud collaborative rendering architecture is to divide the rendering task into two parts: the cloud server and the local terminal device are each responsible for a part of the rendering task.

[0098] The ways to divide rendering tasks can be roughly divided into the following three types.

[0099] The first approach involves separating the foreground and background. The foreground is highly random and difficult to predict, but its rendering workload is small, so it is rendered locally. The background is continuous and predictable, but its rendering workload is large, so it is rendered in the cloud. Simultaneously, the cloud can pre-render multiple future video frames based on the user's posture and position changes, and transmit them to the VR terminal device in advance. Therefore, the transmission of the background does not need to consider latency. The subsequent transmission architecture design in this application will be further elaborated based on this rendering task segmentation method.

[0100] The second method is based on human gaze segmentation. Content near the gaze point is rendered locally, while content surrounding the gaze point is rendered in the cloud.

[0101] The third method is a tile-based segmentation approach. First, a complete video frame is divided into several equal-sized rectangular blocks, with the middleware and cloud-based systems each responsible for rendering a portion of these tiles.

[0102] It should be understood that edge-cloud collaborative rendering avoids black borders and distortion because it does not employ asynchronous time warp technology. However, since the terminal still needs to handle some rendering and decoding tasks, it still requires a certain amount of computing power, preventing VR terminal devices from being further lightweight.

[0103] As can be seen from the above, the mobility issue primarily concerns local rendering. Since VR terminal devices may need to connect to servers via wired connections to obtain computing power to complete rendering tasks, VR terminal device users are restricted to a certain area, limiting their mobility to some extent. The latency issue primarily concerns cloud rendering. In cloud rendering, all rendering tasks are performed in the cloud. The VR terminal device first needs to send the user request to the cloud, which then renders the video according to the request, compresses the rendered image, and sends it back to the VR terminal device. After receiving the compressed data, the VR terminal device needs to decompress it to recover the video frames. However, current network performance cannot meet the requirements of cloud rendering systems to complete the entire process of request reporting, rendering, encoding, transmission, and decoding within the MTP latency range. Regarding bandwidth, as VR services evolve towards deeply immersive modes, such as 8K resolution and 120FPS VR high-definition video, the bandwidth requirements of a single user may exceed 1Gbps. When multiple users using VR terminal devices simultaneously initiate requests, it will put enormous pressure on the wireless network, leading to network congestion and end-to-end transmission latency far exceeding MTP latency, thus affecting the user experience.

[0104] This application proposes a communication method based on edge-cloud collaborative rendering. This method is based on an edge-cloud distributed rendering architecture that integrates computing and network perception. The core idea of ​​this application is to construct a computing set by building multiple nodes with computing and communication capabilities. The VR terminal device and the cloud server can allocate corresponding tasks according to the communication quality and computing resources of each node in the computing set. After completing their tasks, each node feeds back the data to the VR terminal device. This application designs different rendering transmission mechanisms based on the different objects perceived by the computing network information of the computing nodes.

[0105] It should be noted that the terminal device in this application can be a VR terminal device, an AR terminal device, or a MR terminal device. This application does not limit the type of device.

[0106] It should also be noted that, in the specific embodiments of this application, a VR terminal device is used as an example to describe the communication method provided in detail. The first communication device in this application can be a computing power node at the edge of the terminal device, or a computing power node capable of providing services to the terminal device; this application does not limit this. The second communication device in this application can be a cloud server (or application layer), a core network device (e.g., a user plane network element UPF), or an access network device (e.g., a gNB). This application will describe in detail the different situations of the second communication device in the above examples with specific embodiments.

[0107] like Figure 1This illustration shows a distributed rendering architecture provided in this application. Taking a foreground and background separation rendering method for the screen to be displayed on the terminal device as an example, the cloud service is responsible for rendering and encoding the background content, while the VR terminal device is responsible for sensing and sending user interaction information. This user interaction information includes changes in the user's posture and / or position using the VR terminal device, and / or user control commands.

[0108] The first communication device completes the decoding task from the cloud server (also known as the application layer) and the rendering task from the VR terminal device respectively. After completion, the data is sent back to the VR terminal device. The VR terminal device then merges all the data to generate the final video image and displays it.

[0109] It should be noted that the first communication device does not necessarily need to complete both the decoding and rendering tasks simultaneously. For example, a node may only be selected by the cloud server to complete the decoding task of the background content, while a node may only be selected to complete the rendering task of the foreground content.

[0110] Regarding the foreground and background content mentioned in this application, these are examples and not limitations. In the embodiments of this application, the terminal device may present a display interface for displaying video or captured images, such as... Figure 7 As shown. It should be understood that the above... Figure 7 The display interface shown is merely illustrative and is not intended to limit the scope of this application. Figure 7 The displayed interface can also be the human-computer interaction interface presented by the terminal device when detecting user operations. This is an example, not a limitation. Figure 7 As shown, it can be considered that Figure 7 The image shows (a) as the foreground content obtained after processing (i.e., the first foreground content is processed to obtain the second foreground content), and (b) as the background content obtained after processing (i.e., the first background content is processed to obtain the second background content), which is what the terminal device receives. Figure 7 After (a) and (b) in the middle, merge to get Figure 7 As shown in (c), and displayed.

[0111] like Figure 2 A schematic diagram of another distributed rendering architecture proposed in this application is shown. Figure 2As can be seen, the VR terminal device (i.e., the terminal device) sends user interaction information to the access network device, the access network device sends the user interaction information to the core network device, the core network device sends the user interaction information to the cloud server, and the cloud server further determines the first background content to be displayed by the VR terminal device based on the user interaction information, and renders and encodes the first background content to obtain the third background content. In other words, the third background content is the background content obtained by rendering and encoding the first background content.

[0112] Optionally, when the cloud server senses the first communication device (such as...) Figure 2 When the cloud server knows the state of node1, node2, ... (in the first communication device), it may divide the rendered background content into multiple blocks of the same size. For example, the cloud server might divide the rendered background content into four blocks of the same size: S1, S2, S3, and S4. The cloud server encodes each block. It can encode each block into a matching bitrate based on the state of the first communication device. A higher bitrate results in a larger amount of data transmitted after encoding, increasing computational demands and thus requiring higher communication quality and computational power from the first communication device. When the cloud server is unaware of the state of the first communication device, it can encode each block into the same or a different bitrate for data transmission.

[0113] The cloud server generates different Internet Protocol (IP) data streams for each block of encoded data and transmits them to the core network equipment. Upon receiving the IP data streams, the core network equipment maps each IP data stream to a corresponding Quality of Service (QoS) data stream and transmits the QoS data stream to the access network equipment. When the cloud server detects the status of the first communication device, the core network equipment and access network equipment only need to perform the corresponding mapping and transmission based on the data transmitted by the cloud server; no additional data processing is required.

[0114] Optionally, when the core network device (UPF) senses the status of the first communication device, the cloud server may divide the rendered first background content into multiple blocks of the same size. For example, the cloud server divides the rendered first background content into four blocks of the same size, including S1, S2, S3, and S4. The cloud server can encode each block into different bitrate versions (for example, block S1 can be encoded into four different bitrate versions: 1Mbps, 2Mbps, 3Mbps, and 4Mbps), and then transmit the IP data streams generated after encoding each block to the core network device. The core network device selects a suitable first communication device to decode the encoded background content. Specifically, the core network device selects a bitrate version that matches the status of the first communication device from the IP data stream received from the cloud server, and then maps it to different QoS data streams for transmission to the access network device. After receiving the QoS data streams, the access network device maps them to different data radio bearers (DRBs) for transmission to the target first communication device.

[0115] Among them, the distributed transmission architecture in which the cloud server transmits background content blocks of different rates to the core network equipment, and the core network equipment perceives the status information of the first communication equipment, can be further subdivided into the following two cases:

[0116] Scenario 1: One-IP Flow -> Multi-QoS Flow. The cloud server divides the rendered background content into different chunks and encodes each chunk at a different bitrate. The resulting data packets are then transmitted to the core network equipment within the same IP data stream. Because different bitrate versions of different chunks are placed in the same IP data stream, chunk identifiers and bitrate identifiers need to be added to the IP data stream to help the core network equipment identify which content needs to be mapped to the QoS data stream from the IP data stream.

[0117] Scenario 2: Multi-IP Flow -> Multi-QoS Flow. The cloud server divides the rendered background content into different blocks and encodes each block according to a different bitrate version. The resulting data packets are then transmitted to the core network equipment in different IP data streams. Since each IP data stream only contains different bitrate versions of the same block, a bitrate identifier needs to be added to each bitrate version to assist the core network equipment in selecting a suitable bitrate from all bitrate versions and mapping it to the QoS data stream.

[0118] In both scenarios described above, the cloud server needs to add application auxiliary information to the IP extension header of the IP data stream transmitted to the core network equipment to help the core network equipment distinguish between different IP data streams. This application auxiliary information may include the block identifier of the IP data stream and the bitrate identifier corresponding to the IP data stream.

[0119] Optionally, when the access network device senses the status of the first communication device, the cloud server may segment the rendered first background content into blocks. The cloud server can encode each block into a different bitrate version, and then transmit the IP data streams generated from the encoded blocks to the core network device. The core network device selects a bitrate version adapted to the status of the first communication device from the IP data stream received from the cloud server, and then maps it to different QoS data streams for transmission to the access network device. The access network device selects a suitable first communication device to decode the encoded background content, and then maps it to different DRBs for transmission to the target first communication device.

[0120] Based on the different ways in which the cloud server transmits background content blocks to the access network device, the distributed transmission architecture for the access network device to perceive the status information of the first communication device can be further subdivided into the following two cases:

[0121] Scenario 1: One-IP Flow -> One-QoS Flow -> Multi-DRB. The cloud server divides the rendered background content into different blocks and encodes each block according to a different rate version. The encoded data packets are then placed in the same IP data stream and transmitted to the core network device. The core network device then maps the received IP data stream to a QoS data stream and transmits this QoS data stream to the access network device. Because the core network device places blocks with different rate versions in the same QoS data stream, the QoS data stream sent by the core network device needs to include block identifiers and rate identifiers to assist the access network device in selecting the appropriate first communication device and mapping the QoS data stream matching the status of the first communication device to the DRB of that first communication device.

[0122] Scenario 2: Multi-IP Flow -> Multi-QoS Flow -> Multi-DRB. The cloud server divides the rendered background content into different blocks and encodes each block according to a different rate version. The encoded data packets are placed in different IP data streams and transmitted to the core network equipment. The core network equipment then maps the received multiple IP streams to multiple QoS data streams and transmits them to the access network equipment. Since each QoS data stream contains different rate identifiers for the same block, the QoS data stream also needs to carry a rate identifier. This rate identifier is used to assist the access network equipment in selecting the appropriate first communication device from all rate versions and mapping the QoS data stream matching the status of the first communication device to the DRB of that first communication device.

[0123] The above Figure 2 The document primarily describes how the cloud server further determines the first background content based on user interaction information, and then renders the rendered first background content. The cloud server performs block encoding on this rendered first background content, places the block-encoded background content in an IP data stream, and transmits it to the core network device. The core network device performs a one-step mapping based on the IP data stream to obtain a QoS data stream, which is then sent to the access network device. The access network device maps the received QoS data stream to the corresponding DRB of the first communication device.

[0124] at the same time, Figure 2 It also introduces how three different devices—cloud server, core network equipment, and access network equipment—perceive the status of the first communication device, and based on the status of the first communication device, further determine whether to transmit the encoded background content to the first communication device capable of demodulating the background content.

[0125] The above Figure 2 The document details the specific data streams used for transmission between cloud servers, core network equipment, and access network equipment. There are also various connection methods between terminal equipment, the first communication equipment, and access network equipment; this application primarily lists the following methods:

[0126] Method 1

[0127] The terminal device and the first communication device each establish connections with the access network device. In this connection method, the terminal device sends user interaction information to the access network device; the first communication device sends second information indicating its own status to the access network device. The access network device then further determines, based on the specific second communication device that senses the first communication device, whether to transmit the user interaction information and the second information to the upper layer. Communication between the terminal device and the first communication device needs to be based on Uu air interface communication.

[0128] Method 2

[0129] The terminal device's edge communication devices (e.g., computing nodes) are grouped into a computing set, from which one is selected as a proxy node. In this scenario, the terminal device can send user interaction information to the proxy node, which then transmits it to the access network device. The access network device, based on the specific second communication device that senses the first communication device, further decides whether to transmit the user interaction information to the upper layer. Each computing node within the computing set also sends its own second information representing its state to the proxy node. The proxy node then sends all the second information from all nodes to the access network device, which, based on the specific second communication device that senses the first communication device, further decides whether to transmit the user interaction information to the upper layer. In this method, the terminal device communicates with each computing node through the proxy node.

[0130] Method 3

[0131] The terminal device and the first communication device can directly communicate via a sidelink. Compared with method 1, this method has lower latency, but the rate limitation between sidelink communication devices may become a bottleneck.

[0132] The above methods describe the connection methods between the terminal device and the access network device, and between the first communication device and the access network device.

[0133] Next, we will further describe how, after the terminal device establishes a connection with the first communication device, the terminal device further divides the foreground content to be rendered into one or more different tasks and transmits them to one or more first communication devices, which then render the first foreground content. Specifically, the terminal device can determine the first instruction information, i.e., the foreground rendering task allocation, in the following ways:

[0134] Method 1

[0135] Based on user interaction information, the terminal device divides the foreground rendering task into multiple tasks, and then sends these sub-tasks to multiple first communication devices. These first communication devices then render the received rendering tasks.

[0136] Method 2

[0137] The terminal device rationally divides the foreground rendering task based on user interaction information and the status of the first communication device (i.e., the second information), and transmits each of the determined sub-tasks to the corresponding first communication device. The first communication device receives the respective rendering task and renders it.

[0138] Method 3

[0139] The terminal device directly sends the foreground content to be rendered to the first communication device of the agent (i.e., the agent node). The agent node further distributes the foreground rendering task according to the size of the foreground content to be rendered and the status of each computing node. That is, the agent node instructs each node on the task of rendering the foreground content.

[0140] It should be understood that after the first communication device completes the relevant rendering and / or decoding tasks, the first communication device will then transmit the data stream to the terminal device (e.g., ...). Figure 3 (See step S308 shown). Similar to the three methods described above, the first communication device can directly transmit the data stream to the terminal device, or it can first transmit it to a proxy node, and then the proxy node transmits it to the terminal device. The terminal device then merges the data stream received from the first communication device into complete video frames for further display.

[0141] It should be noted that the first communication device that completes the rendering task and the first communication device that completes the decoding task in this application may be the same communication device or different communication devices; or, the first communication device that completes the rendering task and the first communication device that completes the decoding task in this application may be the same computing power node or different computing power nodes, and this application does not limit this.

[0142] Figure 3 A schematic diagram of a communication method provided according to an embodiment of this application is shown. Figure 3 The method shown includes the following steps.

[0143] S301, the terminal device sends a first instruction message to the first communication device.

[0144] Accordingly, the first communication device receives the first instruction information from the terminal device.

[0145] The first instruction information is used to instruct the first communication device to render the first foreground content, which is the foreground content to be displayed by the terminal device.

[0146] It should be understood that the screen to be displayed by the terminal device includes foreground content and background content, wherein the foreground content to be displayed is referred to as "first foreground content" in this application, and the background content to be displayed is referred to as "first background content" in this application.

[0147] It should also be understood that the screen to be displayed in this application may be the original rendered material obtained by the terminal device, or an unprocessed screen.

[0148] Optionally, the terminal device determines the first indication information based on user interaction information and second information. The second information is used to indicate the status of the first communication device and may include one or more of the following: the channel status of the first communication device, the total computing resources of the first communication device, the available computing resources of the first communication device, the remaining power of the first communication device, or the storage space of the first communication device.

[0149] The remaining power of the first communication device refers to the amount of power remaining when the first communication device completes other tasks.

[0150] It should be understood that the terminal device can determine the first foreground content to be rendered based on user interaction information, then determine the capability of the first communication device to render the first foreground content based on the status of the first communication device, and finally send the first instruction information to the first communication device. When the terminal device determines that the rendering of the first foreground content is to be completed by multiple first communication devices (i.e., multiple computing nodes), the second information may include the channel status, total computing resources, available computing resources, remaining power, or storage space of the multiple first communication devices. Based on the first foreground content and the status of the multiple first communication devices, the terminal device further determines the task allocation for rendering the first foreground content and sends the first instruction information to the first communication device.

[0151] It should also be understood that the terminal device can divide the first foreground content to be rendered into different sub-tasks based on user interaction information. The terminal device assigns these divided sub-tasks of rendering the first foreground content to different first communication devices. These multiple first communication devices render different foreground contents simultaneously, thereby shortening the total time for rendering the foreground content to be displayed on the terminal device, reducing latency, improving rendering efficiency, and improving rendering quality, thereby further enhancing the user experience.

[0152] It should be noted that the first communication device can be a computing node, which can also be a computer, mobile phone, monitor, router in a home network, or other VR devices. When the first communication device has excess computing power, it needs to register its computing power with the core network, application server, or access network equipment. The registered first communication device can be used to complete corresponding computing tasks, which may include, but are not limited to, encoding calculations, content rendering, decoding calculations, or encryption.

[0153] S302, the first communication device renders the first foreground content according to the first instruction information to obtain the second foreground content.

[0154] Specifically, after receiving the first instruction information from the terminal device, the first communication device renders the first foreground content according to the first instruction information to obtain the second foreground content, or in other words, the second foreground content is rendered based on the first foreground content.

[0155] It should be noted that the first instruction information may carry the first foreground content. After receiving the first instruction information, the first communication device renders the foreground content based on the first instruction information. Alternatively, the first instruction information and the first foreground content may be transmitted to the first communication device as separate pieces of information. After receiving both the first foreground content and the first instruction information, the first communication device renders the first foreground content based on the first instruction information. Or, the first instruction information may be received by the first communication device from the terminal device, and the first foreground content may be determined by the first communication device based on the acquired user interaction information. This application does not limit the scope of these possibilities.

[0156] It should also be noted that this first instruction information can also be sent from the terminal device to the first communication device in other forms. For example: Method 1. When both the terminal device and the first communication device already have a library of materials for rendering the foreground content, the terminal device only needs to instruct the first communication device on the material number, rendering parameters, and other information. Method 2. When the first communication device does not have a rendering material library, the terminal device needs to send the material to be rendered, rendering parameters, and other information to the first communication device.

[0157] S303, the terminal device sends user interaction information to the second communication device, which is used to determine the first background content.

[0158] The terminal device can send the user interaction information to the second communication device, and the second communication device receives the user interaction information from the terminal device.

[0159] Optionally, the user interaction information may include one or more of the following: changes in the user's posture on the terminal device, changes in the user's position on the terminal device, or user control commands on the terminal device.

[0160] The user control command can be a control command issued by the user through the controller, or a control command issued by the user through other means; this application does not limit this.

[0161] S304, the second communication device receives user interaction information from the terminal device.

[0162] Specifically, the terminal device sends user interaction information to the second communication device, and correspondingly, the second communication device receives user interaction information from the terminal device.

[0163] Optionally, the second communication device determines the first background content to be displayed on the terminal device based on the user interaction information.

[0164] The second communication device further determines the third background content based on the first background content.

[0165] Optionally, the third background content can be rendered and encoded by the second communication device based on the first background content; alternatively, the third background content can also be obtained by the second communication device from locally cached background content based on user interaction information or the first background content. For detailed explanation, please refer to [link to detailed explanation]. Figure 4 The details of steps S404, S405, and S404', S405' are not repeated here.

[0166] It's important to note that receiving user interaction information and sensing the status of the first communication device (i.e., acquiring the second information) are two decoupled processes. The second information can be sensed and acquired by the cloud server, access network device, and core network device respectively. User interaction information is typically sent from the terminal device to the cloud server, which then renders the background content based on this information. The cloud server then transmits the rendered and encoded content to the access network device and core network device. During transmission, the access network device or core network device may establish a local cache table to store the mapping relationship between frequently requested background content and its user interaction information. Thus, upon receiving user interaction information, the access network device or core network device can directly compare the user interaction information with the existing background content in the cache table. If a mapping relationship exists, the cached background content can be directly sent to the first communication device, eliminating the need to upload the user interaction information to the cloud server again.

[0167] like Figure 3 As shown, the method also includes:

[0168] S305, the second communication device sends a second instruction message to the first communication device, or in other words, the first communication device receives a second instruction message from the second communication device.

[0169] Specifically, the second communication device can determine the first background content based on user interaction information, and render and encode the first background content, i.e., the third background content determined after rendering and encoding the first background content; or, the second communication device can determine the first background content based on user interaction information, and obtain the rendered and encoded third background content from the locally cached background content based on the first background content, i.e., the third background content determined after rendering and encoding the first background content; or, the second communication device has a mapping relationship between the user interaction information and the third background content in its local cache table, and the second communication device can also directly obtain the corresponding third background content based on the user interaction information. The second communication device sends the second indication information to the first communication device, which instructs the first communication device to decode and determine the second background content based on the third background content.

[0170] It should be noted that when the access network device is the second communication device, the access network device senses the first communication device and sends the second indication information to the first access network device; when the core network device is the second communication device, the core network device senses the first communication device and sends the second indication information to the first access network device; when the cloud server is the second communication device, the cloud server senses the first communication device and sends the second indication information to the first access network device.

[0171] Optionally, the second communication device determines the second indication information based on the second information and sends the second indication information to the first communication device, wherein the second information is used to indicate the status of the first communication device.

[0172] Optionally, the second information includes one or more of the following: the channel status of the first communication device, the total computing resources of the first communication device, the available computing resources of the first communication device, the remaining power of the first communication device, or the storage space of the first communication device.

[0173] Optionally, the second communication device determines the second indication information based on the second information and the third information, and sends the second indication information to the first communication device, wherein the third information includes the bit rate identifier of the third background content.

[0174] Optionally, the third information may also include block identifiers for the third background content.

[0175] S306, the first communication device determines the second background content based on the second instruction information.

[0176] Specifically, after the first communication device receives the second instruction information from the second communication device, the first communication device decodes the third background content according to the second instruction information to obtain the second background content.

[0177] S307, the terminal device receives first information, which includes second foreground content and second background content.

[0178] Accordingly, the first communication device sends the first information to the terminal device.

[0179] Specifically, the first communication device renders the first foreground content according to the first instruction information to obtain the second foreground content, the first communication device decodes the third background according to the second instruction information of the second communication device to obtain the second background content, and the first communication device sends the second foreground content and the second background content as the first information to the terminal device.

[0180] It should be understood that the first communication device that determines the second foreground content and the first communication device that determines the second background content may be one or more, and this application does not limit this; the second foreground content and the second background content may be sent to the terminal device in the same message or may be sent to the terminal device in different messages, and this application does not limit this; the second foreground content and the second background content may be sent simultaneously or sequentially, and this application does not limit this.

[0181] After the terminal device receives the first information from the first communication device, such as Figure 3 The method shown further includes:

[0182] S308, the terminal device merges the second background content and the second foreground content according to the first information and displays them.

[0183] Specifically, the synthesis of the final content displayed by the terminal device is based on the first communication device completing the rendering of the first foreground content and / or the decoding of the encoded background content. After this process, the first communication device sends a data stream to the terminal device, which then synthesizes the final video and displays it on the terminal device.

[0184] According to the above Figure 3 The solution shown in this application involves a first communication device assisting a terminal device in rendering the first foreground content and decoding the encoded background content. Compared to existing technologies, this eliminates the rendering and decoding tasks for the terminal device itself, reducing its computational load, achieving a lightweight design, and improving the user experience. Furthermore, the first communication device may include multiple computing nodes, allowing rendering and decoding tasks to be distributed across these nodes simultaneously, reducing latency while meeting the high computing power requirements of the terminal device's business.

[0185] Figure 4 A schematic block diagram of another communication method provided in an embodiment of this application is shown below. Figure 4This document details how, when the second communication device is a cloud server, core network device, or access network device, the second communication device determines the third background content and sends a second instruction message to the first communication device. This second instruction message instructs the first communication device to decode the third background content. Figure 4 As shown, the method includes the following steps:

[0186] S401, the terminal device sends a first instruction message to the first communication device.

[0187] Accordingly, the first communication device receives the first instruction information from the terminal device.

[0188] The first instruction information is used to instruct the first communication device to render the first foreground content, which is the foreground content to be displayed by the terminal device.

[0189] S402, the first communication device renders the first foreground content according to the first instruction information to obtain the second background content.

[0190] Specifically, after receiving the first instruction information from the terminal device, the first communication device renders the first foreground content according to the first instruction information to obtain the second foreground content, or in other words, the second foreground content is rendered from the first foreground content.

[0191] S403, the terminal device sends user interaction information to the second communication device.

[0192] Specifically, when the terminal device determines that the user's position, posture, or control commands have changed, the terminal device determines the user interaction information based on the changes.

[0193] Optionally, the user interaction information may include one or more of the following: changes in the user's posture on the terminal device, changes in the user's position on the terminal device, or user control commands on the terminal device.

[0194] Specifically, when the second communication device is an access network device, a core network device, or a cloud server, the second communication device receives user interaction information from the terminal device and further determines the third background content based on the user interaction information. The second communication device determines the second instruction information based on the third background content and sends the second instruction information to the first communication device.

[0195] When the second communication device is a cloud server, such as Figure 4 The method shown further includes:

[0196] S404, the cloud server determines the first background content based on user interaction information, and then determines the third background content based on the first background content.

[0197] Specifically, when the second communication device is a cloud server (or application layer), one possible implementation is as follows: the terminal device first sends the user interaction information to the access network device. When the access network device determines that the cloud server is aware of the state of the first communication device, the access network device sends the user interaction information to the core network device. The core network device then sends the user interaction information to the cloud server. The cloud server further determines the first background content based on the user interaction information and renders and encodes the first background content to obtain the third background content. In other words, the third background content is the background content obtained by rendering and encoding the first background content.

[0198] S405, the cloud server sends a second instruction message to the first communication device.

[0199] Specifically, the cloud server determines the first background content based on user interaction information, and renders and encodes the first background content, resulting in the third background content determined after rendering and encoding the first background content. The cloud server then sends the second instruction information to the first communication device, which instructs the first communication device to decode and determine the second background content based on the third background content.

[0200] Optionally, when the cloud server senses the status of the first communication device, the cloud server determines the second indication information based on the second information and sends the second indication information to the first communication device, wherein the second information is used to indicate the status of the first communication device.

[0201] Optionally, the second information includes one or more of the following: the channel status of the first communication device, the total computing resources of the first communication device, the available computing resources of the first communication device, the remaining power of the first communication device, or the storage space of the first communication device.

[0202] Optionally, the cloud server determines the second indication information based on the second information and the third information, and sends the second indication information to the first communication device, wherein the third information includes the bitrate identifier of the third background content.

[0203] Optionally, the third information may also include block identifiers for the third background content.

[0204] When the second communication device is a core network device or an access network device, such as Figure 4 The method shown further includes:

[0205] S404`, the access network device or core network device obtains third background content from the locally cached background content based on user interaction information.

[0206] Specifically, when the second communication device is an access network device (e.g., gNB) or a core network device (e.g., UPF), one possible implementation is as follows: The terminal device first sends the user interaction information to the access network device. The access network device first checks whether the corresponding background content (i.e., the third background content) has been cached in its local cache. If so, the access network device further determines the second indication information based on the status of the first communication device (i.e., the second information). If not, the access network device sends the user interaction information to the core network device. The core network device then further determines whether it has cached the corresponding background content (i.e., the third background content) locally. If so, the core network device further determines the second indication information. If not, the core network device then sends the user interaction information to the cloud server. The cloud server further determines the first background content based on the user interaction information, and renders and encodes the first background content to determine the third background content.

[0207] The core network or access network equipment can establish a local cache to store background content of different bitrate versions locally and create a corresponding data mapping table. After receiving user interaction information from a terminal device, the core network or access network equipment first checks if the corresponding background content is cached locally. If cached background content is found, the appropriate bitrate version is selected directly based on the second information from the first communication device. If no cached background content is found, the user interaction information is transmitted to the cloud server, which further determines the first background content and performs rendering and encoding. To enable the core network and access network equipment to recognize user interaction information from terminal devices, the MEC (Multi-access Edge Computing) can be deployed to the access network or core network equipment. When multiple users request content from a terminal device, caching frequently accessed content in the access network and core network equipment helps to further reduce end-to-end video transmission latency and improve efficiency.

[0208] It should be noted that after the access network device or core network device receives user interaction information, it can directly obtain the corresponding third background content from the locally cached background content based on the user interaction information; or, after the access network device or core network device receives user interaction information, it can first determine the first background content based on the user interaction information, and then obtain the corresponding third background content from the locally cached background content based on the first background content.

[0209] S405`, the access network device or core network device sends a second instruction message to the first communication device.

[0210] Specifically, when the access network device senses the status of the first communication device, it can further determine the second indication information based on the second information and send the second indication information to the first communication device. When the core network device senses the status of the first communication device, it can further determine the second indication information based on the second information and send the second indication information to the first communication device.

[0211] Optionally, the second information includes one or more of the following: the channel status of the first communication device, the total computing resources of the first communication device, the available computing resources of the first communication device, the remaining power of the first communication device, or the storage space of the first communication device.

[0212] Optionally, the access network device or the core network device determines the second indication information based on the second information and the third information, and sends the second indication information to the first communication device, wherein the third information includes the bit rate identifier of the third background content.

[0213] Optionally, the third information may also include block identifiers for the third background content.

[0214] like Figure 4 The method shown further includes:

[0215] S406, the first communication device determines the second background content based on the second instruction information.

[0216] Specifically, after the first communication device receives the second instruction information from the second communication device, the first communication device decodes the third background content according to the second instruction information to obtain the second background content.

[0217] S407, the terminal device receives first information, which includes second foreground content and second background content.

[0218] Accordingly, the first communication device sends the first information to the terminal device.

[0219] Specifically, the first communication device renders the first foreground content according to the first instruction information to obtain the second foreground content, the first communication device decodes the third background according to the second instruction information of the second communication device to obtain the second background content, and the first communication device sends the second foreground content and the second background content as the first information to the terminal device.

[0220] S408, the terminal device merges the second background content and the second foreground content according to the first information, and displays them.

[0221] Specifically, the synthesis of the final content displayed by the terminal device is based on the first communication device completing the rendering of the first foreground content and / or the decoding of the encoded background content. After that, the first communication device transmits the data stream to the terminal device, and the terminal device then synthesizes the final video and presents it on the terminal device.

[0222] The above Figure 4 Steps S401, S402, S403, S406, S407, and S408 are the same as those described above. Figure 3 Steps S301, S302, S303, S306, S307, and S308 shown are similar; please refer to the above for details. Figure 3 The detailed explanations in the text are omitted here.

[0223] According to the above Figure 4 The scheme shown details how, when the second communication device is an access network device, a core network device, or a cloud server, the second communication device receives user interaction information from the terminal device and determines the possible implementation methods of the third background content based on this user interaction information. Specifically, the access network device and the core network device can establish a local cache to store background content of different rate versions locally and establish a corresponding data mapping table. When the core network device and the access network device receive user interaction information, they first check whether the corresponding background content is already cached locally. If it is, they directly determine the third background content based on the locally cached background content; otherwise, they continue to transmit the user interaction information. This application... Figure 4 The method shown can cache frequently accessed content on core network devices or access network devices when terminal devices receive a large number of requests simultaneously, which helps to further reduce end-to-end data transmission latency and improve user experience.

[0224] The above description primarily focuses on the interaction between devices in this application. It is understood that each device, such as the first communication device and the second communication device, includes at least one of the hardware structures and software modules corresponding to the execution of each function in order to achieve the aforementioned functions. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0225] This application embodiment can divide the first communication device and the second communication device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0226] The above combination Figures 1 to 4 This application provides a detailed description of a communication method based on its embodiments. The following will combine... Figure 5 and Figure 6 This application provides a detailed description of a communication device provided in its embodiments.

[0227] the following Figure 5 and Figure 6 This application provides a detailed description of a communication device according to its embodiments. It should be understood that the descriptions of the device embodiments correspond to the descriptions of the method embodiments; therefore, details not described in detail can be found in the above method embodiments. For brevity, some details are omitted.

[0228] Figure 5 This application provides a schematic block diagram of a communication device 500. The device 500 includes a transceiver unit 510 and a processing unit 520. The transceiver unit 510 can be used to implement corresponding communication functions. The transceiver unit 510 can also be referred to as a communication interface or communication unit. The processing unit 520 can be used to implement corresponding processing functions, such as modifying addresses.

[0229] Optionally, the device 500 further includes a storage unit, which can be used to store instructions and / or data. The processing unit 520 can read the instructions and / or data in the storage unit so that the device can perform the operation of the device or network element in the foregoing method embodiments.

[0230] In the first design, the device 500 can be the terminal device in the foregoing embodiments, or it can be a component of the terminal device (such as a chip). The device 500 can implement the steps or processes corresponding to those executed by the terminal device in the above method embodiments, wherein the transceiver unit 510 can be used to perform the transceiver-related operations of the terminal device in the above method embodiments, and the processing unit 520 can be used to perform the processing-related operations of the terminal device in the above method embodiments.

[0231] In one possible implementation, the transceiver unit 510 is configured to send first instruction information to the first communication device. The first instruction information instructs the first communication device to render first foreground content, which is the foreground content to be displayed by the terminal device. The transceiver unit 510 is also configured to receive first information from the first communication device. The first information includes second foreground content and second background content, wherein the second foreground content is rendered based on the first foreground content, the second background content is decoded based on third background content, and the third background content is determined based on the first background content. The first background content is the background content to be displayed by the terminal device.

[0232] Optionally, the transceiver unit 520 is also used to send user interaction information to the second communication device, the user interaction information being used to determine the first background content.

[0233] Optionally, the third background content is rendered and encoded by the second communication device based on the first background content.

[0234] Optionally, the third background content is obtained by the second communication device from the locally cached background content based on the first background content.

[0235] Optionally, the processing unit 510 is configured to determine first indication information based on user interaction information and second information, wherein the second information is used to indicate the status of the first communication device.

[0236] Optionally, the second information includes one or more of the following: the channel status of the first communication device, the total computing resources of the first communication device, the available computing resources of the first communication device, the remaining power of the first communication device, or the storage space of the first communication device.

[0237] Optionally, user interaction information includes one or more of the following: changes in the user's posture, changes in the user's position, or user control commands.

[0238] In the second design, the device 500 can be the first communication device in the foregoing embodiments, or it can be a component of the first communication device (such as a chip). The device 500 can implement the steps or processes corresponding to those performed by the first communication device in the above method embodiments, wherein the transceiver unit 510 is used to perform the transceiver-related operations of the first communication device in the above method embodiments, and the processing unit 520 is used to perform the processing-related operations of the first communication device in the above method embodiments.

[0239] In one possible implementation, the transceiver unit 510 is used to receive first instruction information from the terminal device. The first instruction information is used to instruct the first communication device to render first foreground content, and the first foreground content is the foreground content to be displayed by the terminal device.

[0240] The transceiver unit 510 is also configured to receive second instruction information from the second communication device. The second instruction information is used to instruct the first communication device to decode the third background content. The third background content is determined based on the first background content, and the first background content is the background content to be displayed by the terminal device.

[0241] The transceiver unit 510 is also used to send first information to the terminal device. The first information includes second foreground content and second background content, wherein the second foreground content is rendered based on the first foreground content and the second background content is decoded based on the third background content.

[0242] Optionally, the third background content is rendered and encoded by the second communication device based on the first background content.

[0243] Optionally, the third background content is obtained by the second communication device from the locally cached background content based on the first background content.

[0244] Optionally, the transceiver unit 510 is also used to send second information, which is used to indicate the status of the first communication device.

[0245] Optionally, the second information includes one or more of the following: the channel status of the first communication device, the total computing resources of the first communication device, the computable resources of the first communication device, the remaining power of the first communication device, or the storage space of the first communication device.

[0246] In the third design, the device 500 can be the second communication device in the foregoing embodiments, or it can be a component of the second communication device (such as a chip). The device 500 can implement the steps or processes performed by the second communication device corresponding to the method embodiments described above, wherein the transceiver unit 510 is used to perform the transceiver-related operations of the second communication device in the method embodiments described above, and the processing unit 520 is used to perform the processing-related operations of the second communication device in the method embodiments described above.

[0247] In one possible implementation, the transceiver unit 510 is used to receive user interaction information from the terminal device, the user interaction information being used to determine a first background content, the first background content being the background content to be displayed by the terminal device;

[0248] The transceiver unit 510 is also used to send a second instruction information to the first communication device. The second instruction information is used to instruct the first communication device to decode the third background content, which is determined by the second communication device based on the first background content.

[0249] Optionally, the third background content is rendered and encoded by the second communication device based on the first background content.

[0250] Optionally, the third background content is obtained by the second communication device from locally cached background content based on the first background content.

[0251] Optionally, the processing unit 520 is further configured to send second indication information to the first communication device according to the second information, the second information being used to indicate the status of the first communication device.

[0252] Optionally, the second information includes one or more of the following: the channel status of the first communication device, the total computing resources of the first communication device, the available computing resources of the first communication device, the remaining power of the first communication device, or the storage space of the first communication device.

[0253] Optionally, the processing unit 520 is further configured to determine second indication information based on the second information and the third information, wherein the third information includes a bitrate identifier of the third background content.

[0254] Optionally, the third information may also include block identifiers for the third background content.

[0255] Optionally, the user interaction information includes one or more of the following: changes in the user's posture on the terminal device, changes in the user's position on the terminal device, or user control commands on the terminal device.

[0256] It should be understood that the specific process of each unit performing the above-mentioned corresponding steps has been described in detail in the above-mentioned method embodiments, and will not be repeated here for the sake of brevity.

[0257] It should also be understood that the device 500 here is embodied in the form of a functional unit. The term "unit" here can refer to an application-specific integrated circuit (ASIC), electronic circuitry, a processor (e.g., a shared processor, a proprietary processor, or a group processor, etc.) and memory for executing one or more software or firmware programs, combined logic circuitry, and / or other suitable components supporting the described functions. In an alternative example, those skilled in the art will understand that the device 500 may specifically be a terminal device in the above embodiments, used to execute the various processes and / or steps corresponding to the terminal device in the above method embodiments; or, the device 500 may specifically be a first communication device in the above embodiments, used to execute the various processes and / or steps corresponding to the first communication device in the above method embodiments; or, the device 500 may specifically be a second communication device in the above embodiments, used to execute the various processes and / or steps corresponding to the second communication device in the above method embodiments; to avoid repetition, further details are omitted here.

[0258] The apparatus 500 of each of the above-described schemes has the function of implementing the corresponding steps performed by the terminal device, the first communication device, or the second communication device in the above-described methods. The functions can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions; for example, the transceiver unit can be replaced by a transceiver (e.g., the sending unit in the transceiver unit can be replaced by a transmitter, and the receiving unit in the transceiver unit can be replaced by a receiver), and other units, such as processing units, can be replaced by processors, respectively executing the transceiver operations and related processing operations in each method embodiment.

[0259] In addition, the transceiver unit 510 may also be a transceiver circuit (for example, it may include a receiving circuit and a transmitting circuit), and the processing unit may be a processing circuit.

[0260] It should be pointed out that, Figure 5 The device mentioned can be the network element or equipment in the foregoing embodiments, or it can be a chip or a chip system, such as a system on a chip (SoC). The transceiver unit can be an input / output circuit or a communication interface; the processing unit is a processor, microprocessor, or integrated circuit integrated on the chip. No limitations are imposed here.

[0261] like Figure 6 As shown in the illustration, this application provides a schematic block diagram of another communication device. The device 600 includes a processor 610, which executes computer programs or instructions stored in a memory 620, or reads data / signaling stored in the memory 620, to perform the methods described in the above method embodiments. Optionally, there may be one or more processors 610.

[0262] Optionally, such as Figure 6 As shown, the device 600 also includes a memory 620 for storing computer programs or instructions and / or data. The memory 620 may be integrated with the processor 610 or may be disposed separately. Optionally, there may be one or more memories 620.

[0263] Optionally, such as Figure 6 As shown, the device 600 also includes a transceiver 630 for receiving and / or transmitting signals. For example, a processor 610 controls the transceiver 630 to receive and / or transmit signals.

[0264] As one option, the device 600 is used to implement the operations performed by the various devices in the above method embodiments.

[0265] For example, processor 610 is used to execute computer programs or instructions stored in memory 620 to implement the relevant operations of the terminal device in the various method embodiments described above. For example, Figure 3 The operation performed by the terminal device in any of the implementation methods.

[0266] It should be understood that the processor mentioned in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0267] It should also be understood that the memory mentioned in the embodiments of this application can be volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM). For example, RAM can be used as an external cache. By way of example and not limitation, RAM includes the following forms: static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0268] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) can be integrated into the processor.

[0269] It should also be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0270] This application also provides a computer-readable storage medium storing computer instructions for implementing the methods executed by the network element in the above-described method embodiments.

[0271] For example, when the computer program is executed by the computer, it enables the computer to implement the methods executed by the terminal device in the various embodiments of the above methods.

[0272] For example, when the computer program is executed by the computer, it enables the computer to implement the methods executed by the first communication device in the various embodiments of the above methods.

[0273] For example, when the computer program is executed by the computer, it enables the computer to implement the methods executed by the second communication device in the various embodiments of the above methods.

[0274] This application also provides a computer program product comprising instructions which, when executed by a computer, implement the methods performed by a terminal device or communication device in the above-described method embodiments.

[0275] This application also provides a communication system, including one or more of the aforementioned terminal device, first communication device, and second communication device.

[0276] The explanations and beneficial effects of the relevant contents in any of the devices provided above can be found in the corresponding method embodiments provided above, and will not be repeated here.

[0277] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed between each other may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0278] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. For example, the computer can be a personal computer, a server, or a network device, etc. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks, SSDs). For example, the aforementioned available media include, but are not limited to, USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, and other media capable of storing program code.

[0279] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0280] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0281] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0282] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0283] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0284] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0285] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A communication method characterized by comprising: include: The terminal device sends a first instruction message to the first communication device. The first instruction message is used to instruct the first communication device to render a first foreground content, which is the foreground content to be displayed by the terminal device. The terminal device receives first information from the first communication device. The first information includes second foreground content and second background content. The second foreground content is rendered based on the first foreground content, and the second background content is decoded based on a third background content. The third background content is determined based on the first background content, and the first background content is the background content to be displayed by the terminal device. The terminal device sends user interaction information to the second communication device, and the user interaction information is used to determine the first background content.

2. The method according to claim 1, characterized in that, The third background content is obtained by the second communication device rendering and encoding based on the first background content.

3. The method according to claim 1, characterized in that, The third background content is obtained by the second communication device from the locally cached background content based on the first background content.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: The terminal device determines the first indication information based on the user interaction information and the second information, wherein the second information is used to indicate the status of the first communication device. The second information includes one or more of the following: The channel status of the first communication device, the total computing resources of the first communication device, the available computing resources of the first communication device, the remaining power of the first communication device, and the storage space of the first communication device.

5. The method according to any one of claims 1 to 3, characterized in that, The user interaction information includes one or more of the following: The user's posture changes, the user's position changes, and the user's control commands on the terminal device.

6. A communication method, characterized in that, include: The first communication device receives first instruction information from the terminal device. The first instruction information is used to instruct the first communication device to render first foreground content, which is the foreground content to be displayed by the terminal device. The first communication device receives a second instruction from the second communication device. The second instruction is used to instruct the first communication device to decode a third background content. The third background content is determined based on the first background content, which is the background content to be displayed by the terminal device. The first communication device sends first information to the terminal device, the first information including second foreground content and second background content. The second foreground content is rendered based on the first foreground content, and the second background content is decoded based on the third background content. The third background content is rendered and encoded by the second communication device based on the first background content.

7. The method according to claim 6, characterized in that, The third background content is obtained by the second communication device from the locally cached background content based on the first background content.

8. The method according to claim 6 or 7, characterized in that, The method further includes: The first communication device sends a second message, which indicates the status of the first communication device. The second information includes one or more of the following: The channel status of the first communication device, the total computing resources of the first communication device, the computable resources of the first communication device, the remaining power of the first communication device, and the storage space of the first communication device.

9. A communication method, characterized in that, include: The second communication device receives user interaction information from the terminal device, the user interaction information being used to determine a first background content, the first background content being the background content to be displayed by the terminal device; The second communication device sends a second instruction message to the first communication device. The second instruction message is used to instruct the first communication device to decode the third background content. The third background content is determined by the second communication device based on the first background content. The third background content is rendered and encoded by the second communication device based on the first background content.

10. The method according to claim 9, characterized in that, The third background content is obtained by the second communication device from the locally cached background content based on the first background content.

11. The method according to claim 9 or 10, characterized in that, The second communication device sends the second indication information to the first communication device, including: The second communication device sends the second indication information to the first communication device based on the second information, wherein the second information is used to indicate the status of the first communication device. The second information includes one or more of the following: The channel status of the first communication device, the total computing resources of the first communication device, the available computing resources of the first communication device, the remaining power of the first communication device, or the storage space of the first communication device.

12. The method according to claim 11, characterized in that, The second communication device sends the second instruction information to the first communication device based on the second information, including: The second communication device sends the second instruction information to the first communication device based on the second information and the third information, wherein the third information includes the bitrate identifier of the third background content.

13. The method according to claim 12, characterized in that, The third information also includes the block identifiers of the third background content.

14. The method according to claim 9 or 10, characterized in that, The user interaction information includes one or more of the following: The user's posture change, the user's position change, or the user's control command of the terminal device.

15. A communication apparatus, characterized in that, The apparatus includes: a unit for performing the method as described in any one of claims 1 to 5, or a unit for performing the method as described in any one of claims 6 to 8, or a unit for performing the method as described in any one of claims 9 to 14.

16. A communication apparatus, characterized in that, include: A processor for executing a computer program stored in a memory to cause the apparatus to perform the method as claimed in any one of claims 1 to 5, or to cause the apparatus to perform the method as claimed in any one of claims 6 to 8, or to cause the apparatus to perform the method as claimed in any one of claims 9 to 14.

17. The apparatus according to claim 16, characterized in that, The device also includes the memory.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed on a computer, causes the computer to perform the method as described in any one of claims 1 to 5, or causes the computer to perform the method as described in any one of claims 6 to 8, or causes the computer to perform the method as described in any one of claims 9 to 14.

19. A computer program product, characterized in that, The computer program product includes instructions for performing the method as described in any one of claims 1 to 5, or the computer program product includes instructions for performing the method as described in any one of claims 6 to 8, or the computer program product includes instructions for performing the method as described in any one of claims 9 to 14.

20. A communication system, characterized in that, Includes terminal equipment, a first communication device, and a second communication device; The terminal device is used to perform the method as described in any one of claims 1 to 5; The first communication device is used to perform the method as described in any one of claims 6 to 8; The second communication device is used to perform the method as described in any one of claims 9 to 14.

Images